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Chapter  1 


GENERAL 


1.1  PREFACE 


The  primary  measure  of  success  for  a  weapon  system  is  that 
it  worked  well  when  it  was  fielded.— ^  However,  the  keys  to 
successful  program  management  are  control  of  cost,  schedule, 
technical  performance  including  supportability  and  the  coordina¬ 
tion  of  the  new  systems  into  the  structure  of  the  sponsoring 
military  service.  The  ability  to  effectively  and  efficiently 
trade-off  between  these  elements  is  crucial  in  today's  complex 
program  acquisition  management  environment.  To  make  these 
required  trade-offs  (less  the  coordination  into  the  service 
structure  which  will  not  be  addressed  here),  the  Program  Manager 
must  know  the  status  of  each  of  the  keys  as  well  as  the  impact 
that  a  change  in  any  one  has  on  the  others.  A  series  of  Depart¬ 
ment  of  Defense  (DoD)  criteria  has  been  developed  to  ensure  that 
contractors'  management  information  systems  will  provide  valid 
cost  and  schedule  performance  data.  The  measurement  of  the 
contractor's  technical  performance  progress  toward  the  accom¬ 
plishment  of  the  required  performance  specifications  is  moni¬ 
tored  through  a  combination  of  methods.  It  is  the  intent  of 
this  handbook  to  illuminate  these  methods  for  the  Government 
Program  Office  thereby  assisting  in  the  difficult  task  of  field¬ 
ing  weapon  systems  that  are  affordable  and  work  well. 

1.2  HISTORY 


The  use  of  Cost/Schedule  Control  Systems  Criteria  (C/SCSC) 
is  now  well  instituted  within  the  DoD  and  is  also  being  applied 
by  a  number  of  other  Federal  agencies.  The  criteria  have 
evolved  over  the  years  and  are  still  subject  to  various  inter- 


1/  "Managing  for  Success  in  Defense  Systems  Acquisition" 
Baumgartner,  Brown,  &  Kelley  (DSMC). 


pretations.  The  development  of  C/SCSC  goes  back  to  the  late 
1950's  and  early  1960's  and  the  Polaris  Missile  Program.  The 
Polaris  Program  Management  Office  developed  a  management  infor¬ 
mation  system  that  could  be  used  to  measure  contractor  perform¬ 
ance  throughout  the  course  of  the  program.  This  system  subse¬ 
quently  became  the  Program  Evaluation  and  Review  Technique 
(PERT).  It  is  a  network  scheduling  technique  that  graphically 
displays  the  interrelationships  of  specific  program  activities 
and  establishes  the  critical  paths  or  path  through  the  network 
on  which  management  should  focus  its  attention.  A  capability  to 
budget  and  report  costs  by  PERT  was  developed  and  named  PERTCOST. 
By  the  mid  1960's  several  variations  of  PERTCOST  existed  in  DoD, 
and  monthly  cost  reports  were  required.  PERTCOST  requirements 
were  negotiated  into  contracts  even  when,  in  some  cases,  con¬ 
tractors  had  equally  effective  alternate  systems  in  place.  This 
required  operation  of  separate  systems:  one  used  for  government 
reporting  and  one  used  by  the  contractor  for  actual  management. 

The  PERTCOST  system  in  some  cases  called  for  the  reporting 
of  costs  by  the  lowest  level  network  activities.  This  resulted 
in  the  collection  of  large  amounts  of  very  expensive  detailed 
cost  information  which  was  not  consistently  utilized. 

At  the  same  time,  the  Air  Force  Minuteman  Missile  Program 
Office,  with  contractor  support,  developed  a  contractor  Per¬ 
formance  Measurement  System  based  on  the  lessons  learned  from 
PERTCOST.  It  utilized*  the  work  breakdown  structure  and  work 
packages  of  PERTCOST.  Work  accomplished  was  measured  in  rela¬ 
tion  to  the  budget  planned  for  the  work;  thus,  the  name  Earned 
Value . 


A  second  Air  Force  effort  was  established  in  the  Office  of 
the  Secretary  of  the  Air  Force.  This  effort  developed  a  set  of 
simplified  standards  by  which  to  measure  a  contractor's  internal 
management  system.  The  standards  contained  the  key  elements  of 
PERTCOST  and  the  Earned  Value  system  and  eliminated  the  detailed 


cost  reporting  from  PERT  networks.  This  system  was  published  by 
Air  Force  System  Command  in  1966  as  Cost/Schedule  and  control 
Specification  (C  Spec).  ' 

In  December  1967,  DoD  Instruction  7000.2,  Performance  Meas¬ 
urement  of  Selected  Acquisitions,  was  published  by  the  Assistant 
Secretary  of  Defense  (Comptroller)  and  authorized  the  publishing 
of  a  Tri-Service  C/SCSC  Joint  Implementation  Guide.  The  guide 
provides  the  formal  material  for  implementation  of  the  crite¬ 
ria.  It  was  published  in  August  1970  and  has  had  three  updates 
thru  October  1980. 

Throughout  the  early  development  of  C/SCSC,  technical  per¬ 
formance  was  envisioned  as  a  part  of  the  criteria;  however,  in 
1967  a  Technical  Performance  Measurement  (TPM)  system  began 
emerging  as  a  separate  requirement  within  DoD.  Engineering  was 
within  the  domain  of  the  Director  of  Defense  Research  and  Engi¬ 
neering  ( DDR&E ) ,  at  the  Office  of  the  Secretary  of  Defense  (OSD) 
level,  and  the  responsibility  of  the  Ass't  Secretary  (R&D), 
within  the  services.  This  split  in  functions  between  the  Com¬ 
ptroller  (C/SCSC)  and  DDR&E  (TPM)  at  the  OSD  level  appears  to 
have  been  the  prime  reason  for  the  development  of  entirely 
separate  guidance  documents  for  TPM  and  C/SCSC.  TPM  was  for¬ 
mally  introduced  by  MIL-STD-499  (USAF)  in  July  1967  (superseded 
in  May  1974  by  MIL-STD-499A( USAF] ,  Engineering  Management). 
This  document  was  developed  to  assist  government  and  contractor 
personnel  in  defining  the  system  engineering  effort  required  in 
support  of  defense  acquisition  programs,  and  includes  a  list  of 
specific  Statement  of  Work  (SOW)  clauses  that  can  be  selected 
for  contract  inclusion. 

1.3  PURPOSE 

The  function  of  this  handbook  is  to  provide  program  manage¬ 
ment  personnel  with  insight  into  how  technical  performance  is 
measured  by  the  contractor,  reported  to  the  government  and  how 
this  information  can  be  effectively  integrated  with  cost  and 


schedule  performance  data.  Separate  systems  have  been  developed 
which,  when  properly  linked,  provide  the  keys  to  good  manage¬ 
ment.  Understanding  how  contractors  u*'li2e  their  cost  schedule 
and  TPM  data  to  evaluate  their  own  progress  will  provide  govern¬ 
ment  Program  Managers  insight  into  how  to  monitor  their  contrac¬ 
tors  overall  progress. 

1.4  ORGANIZATION 

Chapters  2  and  3  provide  overviews  of  TPM  and  C/SCSC  taken 
from  published  documentation.  Chapter  4  was  developed  using 
extensive  interviews  with  engineering  management  personnel 
within  government  program  offices  and  DoD  contractors.  It 
illustrates  how  technical,  cost  and  schedule  performance  are 
actually  being  monitored  and  used  to  provide  program  control 
today.  The  differences  between  the  government  program  office 
and  the  contractor  viewpoints  are  highlighted.  Chapter  5  iden¬ 
tifies  the  issues  which  must  be  considered  in  PMO  implementation 
and  execution  of  a  TPM  program. 


Chapter  2 

TECHNICAL  PERFORMANCE  MEASUREMENT 

2.1  INTRODUCTION 

Technical  Performance  Measurement  (TPM)  is  an  integral 
function  of  system  engineering.  For  maximum  utility,  TPM  must 
be  compatible  with  other  related  program  management  activities 
(cost/schedule  control  system  criteria,  contract  administration, 
production  management,  readiness  functions). 

The  means  of  measuring  technical  performance  vary  with  the 
program  phase  from  engineering  analysis  of  design  in  the  early 
stage  of  Pull  Scale  Development  (FSD)  to  system  qualification 
test  in  the  latter  stage  of  FSD.  TPM  is  not  a  substitute  for  a 
risk  management  program,  rather  it  is  a  critical  function  of 
such  a  program.  TPM  along  with  C/SCSC  provide  the  measurement 
function  of  a  risk  management  program.  It  is  the  measurement  of 
progress  of  cost,  schedule  and  technical  performance  against  key 
parameters  that  provides  management  progress  information  and 
problem  identification.  Further,  TPM  is  not  a  process  by  which 
to  evaluate  the  productivity  of  a  single  pt  group  of  design 
engineers,  but  rather  a  process  used  to  evaluate  the  progress  of 
an  organizational  entity  against  established  performance  goals 
for  the  weapon  system's  elements. 

2.2  DEFINITION 

TPM  is  defined  as:  "the  continuing  prediction  and  demonstra¬ 
tion  of  the  degree  of  anticipated  or  actual  achievement  of 
selected  technical  objectives.  It  includes  an  analysis  of  any 
differences  between  the  achievement  to  date,  current  estimate, 
and  the  specific  requirement.  Achievement  to  date  is  the  value 
of  a  technical  parameter  estimated  or  measured  in  a  particular 
test  or  analysis.  Current  estimate  is  the  value  of  a  technical 
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parameter  predicted  to  be  achieved  at  the  end  of  the  contract 
within  existing  resources ^ ^ 

TPM  is;  "...the  design  assessment  that  predicts,  through 
engineering  analysis  or  test  measurements,  the  values  of  essen¬ 
tial  system  performance  parameters. ■( 2) 

Dr.  Norman  Waks,  formerly  of  DDR&E,  defines  TPM  as;  “...the 
regular  demonstration  through  test  or  prediction,  extrapola¬ 
tion,  or  other  forecasting  technique,  of  the  degree  of  actual  or 
anticipated  achievement  of  selected  technical  goals  or 
objectives  of  a  system,  component,  or  equipment  project/program 
and  an  accounting,  in  the  causal  sense,  for  the  difference 
between  the  result  of  this  status  reading  and  that  which  was 
planned,  in  a  fashion  which  permits  appropriate  managers  to  take 
timely  action  on  indicated  problems. * ( 3 ) 

These  definitions  lead  to  two  conclusions,  the  first  is  that 
the  definitions  tend  to  address  a  TPM  program  rather  than  TPM; 
and  the  second  is  that  TPM  lends  itself  to  being  structured  into 
the  following  four  key  elements: 

o  Status  against  a  plan, 
o  Estimate  of  future  attainment, 
o  variance  analysis, 
o  Problem  identification  analysis. 

Performance  measurement  and  forecast  of  key  parameters  must 
be  performed  on  a  regular  basis  to  determine  future  courses  of 
action  in  time  to  be  effective.  The  frequency  of  this  measure¬ 
ment  and  forecast  cycle  is  related  to  the  complexity  of  the 
program.  As  a  minimum,  the  major  established  milestones  such  as 
Preliminary  Design  Review  ( PDR ) ,  Critical  Design  Review  (CDR) , 


(1)  MIL-STD  499A  Engineering  Management  (USAF)  May  1974 

(2)  System  Engineering  Management  Guide,  Defense  Systems  Man¬ 
agement  College,  3  October  1983 

(3)  Waks,  Norman:  Technical  Performance  Measurement  -  A  De¬ 
fense  Department  View,  December  1968 
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etc.,  provide  normal  transitions  in  engineering  activities 
which  require  total  assessment  of  technical  status.  For  most 
complex  developments  it  is  necessary  to  identify  meaningful 
sub-milestones  that  will  force  the  aggregation  of  technical 
information  and  the  conduct  of  meaningful  measurement  and 
forecast. 

2.3  TPM  PROCESS 

2.3.1  General 

Design  and  development  is  an  iterative  process  that  allows 
the  engineer  to  move  from  the  unknown  to  the  known  in  an  effort 
to  define  a  product  that  will  satisfy  stated  requirements. 
Integral  to  any  contractor  design  activity  is  a  continuous 
effort  to  evaluate  proposed  and  revised  designs  so  as  to  pro¬ 
ject  their  expected  performance.  The  head  of  the  design  activ¬ 
ity  is  continually  faced  with  assessing  progress  and  making 
design  alternative  decisions.  The  expected  results  of  these 
decisions  are  increments  in  the  optimization  of  the  system 
design.  This  process  continues  until  either  an  optimum  design 
is  obtained  (as  defined  in  the  contract),  the  originally  allo¬ 
cated  resources  of  time  and  funds  are  expended,  a  revised  level 
of  resources  is  established  and  expended,  or  the  originally 
specified  design  constraints  are  modified  to  coincide  with 
predicted  achievable  levels. 

From  the  senior  OSD  management  viewpoint,  the  systems 
acquisition  life  cycle  is  primarily  a  process  in  risk  manage¬ 
ment,  not  design  selection.  Under  this  philosophy,  a  key 
management  principle  is  the  establishment  of  goals  and  thres¬ 
holds  for  cost,  schedule,  and  performance,  readiness  and  sup- 
portability.  This  is  stated  in  Defense  Circular  #76-43,  pg.  14 
dated  22  March  1983.  Goals  are  values  that  will  enable  the  new 
system  to  fully  satisfy  mission  needs.  Thresholds  are  values 
that  describe  a  minimum  performance  level  or  a  maximum  expendi¬ 
ture  of  resources  for  a  new  system.  Variances  between  goals 


and  thresholds  reasonably  reflect  the  degree  of  risk  in  an 
acquisition  program  at  each  milestone.  Threshold  breaches 
require  a  reassessment  of  the  program  in  terms  of  mission  need 
and  prioritization  among  other  acquisition  programs.  Program 
managers  must  report  actual  or  projected  threshold  breaches  to 
the  Defense  Acquisition  Executive  providing  an  assessment  of 
the  problem  and  recommendations.  An  example  of  a  goal  versus  a 
threshold  would  be: 


Parameter 


Threshold 


Weapon  System  reliability  70%  65% 

Fuel  consumption  (Gal/Hr)  650  700 
Maintenance  Man-hours/Flying  Hour  35  40 

In  theory,  a  properly  structured,  formal  TPM  program  pro¬ 
vides  the  government  Program  Manager  with  the  bridge  between 
the  contractor's  engineering  design  activity  and  the  govern¬ 
ment's  management  objectives.  The  TPM  program  should  identify 
problem  areas  and  the  probable  impact  on  the  acquisition  pro¬ 
gram  by  means  of  assessment  of  the  contractor's  technical 
achievement  trends.  Solutions  to  potentially  unacceptable 
impacts  identified  by  TPM  are  considered  a  management  function 
and  not  a  part  of  the  basic  TPM  process.  Also,  for  the  formal 
TPM  program  to  be  efficient,  credible  and  affordable,  it  should 
be  an  integral  part  of  the  contractor's  engineering  management 
system  and  not  an  additional  management  reporting  structure. 

The  following  sections  discuss  the  attributes  of  a  formal¬ 
ized  TPM  program  and  the  interrelationship  with  the  engineering 
process . 


2.3.2  Period  of  Formalized  TPM 


[•Til 
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assesses  the  system  design  progress  toward  meeting  stated 
mission  requirements.  Therefore,  the  development  of  basic 
performance  requirements  and  the  accumulation  of  deployment 
experience  data  are  not  included  in  TPM.  TPM  is  applicable 
from  the  start  of  subsystem  detail  design  until  release  of  the 
production  baseline  specifications. 

Specifically,  TPM  planning  should  normally  be  accomplished 
in  Demonstration/Validation  with  a  detailed  implementation  plan 
available  for  review  at  the  time  of  the  Systems  Design  Review 
(SDR).  The  allocated  performance  requirements  approved  during 
the  Preliminary  Design  Review  (PDR)  at  the  start  of  FSD  are  the 
baselines  from  which  the  TPM  program  should  assess  progress. 
The  program  is  complete  when  attainment  of  all  performance 
specifications  have  been  demonstrated  by  the  successful  comple¬ 
tion  of  the  Functional  Configuration  Audit  (FCA). 


2.3.3  TPM  Planning 


Within  the  Systems  Engineering  Management  Plan  (SEMP)  the 
contractor  should  address  the  planning  for  TPM.  The  TPM  plan 
should  identify  parameters  to  be  tracked,  the  parameter  pro¬ 
files  with  time  or  standards,  reporting  mechanisms,  analysis 
and  forecast  techniques,  key  test  events,  TPM  report  dates, 
implementation  procedures,  and  information  flow.  The  plan 
should  also  identify  organizational  and  individual  responsi¬ 
bility.  Section  2.4  will  discuss  the  SEMP  format. 


The  degree  of  visibility  into  technical  progress  will  be 
determined  by  the  care  used  in  selecting  the  key  parameters, 
and  the  frequency  and  completeness  of  the  evaluation  and  fore¬ 
cast  effort.  A  properly  structured  systems  engineering  plan 
inherently  provides  a  capability  to  measure  performance  without 
the  need  for  special  additional  reporting  tasks  being  added 
with  their  associated  cost  increases,  since  it  must  address  the 
key  parameters  and  their  balance  in  order  to  perform  the  system 
engineering  function  of  specialty  integration. 
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Figure  1  displays  the  Acquisition  Phases  with  the  normally 
scheduled  reviews  and  audits,  and  the  associated  resultant 
specifications.  A  generalized  schedule  of  available  evaluation 
techniques  is  also  implied  by  the  schedule  of  engineering 
analysis  efforts  and  test  events.  In  major  acquisition  pro¬ 
grams,  the  design  reviews  and  audits  may  not  be  single  sched¬ 
uled  events  but  rather  a  series  of  events,  for  each  major 
configuration  item  (Cl). 

As  an  example,  the  CDR  for  a  new  tactical  fighter  aircraft 
may  be  a  series  of  separate  CDRs  on  the  CIs  such  as  the  engine, 
airframe,  and  avionics;  while  for  a  tactical  missile,  the  CIs 
may  be  the  propulsion  system,  guidance  section,  armament  sec¬ 
tion  and  the  control  section.  In  addition  to  these  normal 
program  events,  there  are  other  identifiable  events,  sub-mile¬ 
stones,  subsystem  status  reviews,  and  critical  item  analysis 
and  performance  tests  that  provide  suitable  points  in  the 
program  where  technical  progress  can  be  assessed.  Therefore, 
the  key  to  adequate  visibility  is  tied  to  the  ability  to  iden¬ 
tify  meaningful  measurement  points  within  the  overall  program 
schedule  between  major  milestones. 

2.3.4  Parameter  Selection 

The  Mission  Area  Analysis  process  identifies  the  system 
level  technical  performance  requirements  and  documents  them  in 
the  System  Specifications.  The  System  Engineering  process 
reduces  these  to  Contract  Items  allocated  or  design  to  Speci¬ 
fications.  Figure  2  illustrates  these  influences  on  the  TPM 
parameter  selection  process.  The  selection  of  parameters  to  be 
tracked  and  reported  is  a  function  of  TPM  and  must  start  in  the 
Demonstration/Validation  phase  to  permit  initiation  with  FSD . 
The  TPM  parameters  are  normally  selected  because  ti.ey  are: 

o  Mission  Critical 

o  State-of-the-Art  Critical 
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Figure  1.  PROGRAM  REVIEWS,  AUDITS,  SPECIFICATIONS 
TESTS,  AND  ENGINEERING  ANALYSIS  BY  PHASE 
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on  of  TPM  Parameters 


A  mission  critical  parameter,  may  be  ship  speed,  range  or 
endurance,  or  it  may  be  the  performance  reliability  or  detec¬ 
tion  range  of  a  subsystem  such  as  radar.  State-of-the-Ar t 
Critical  may  be  the  development  of  a  new  graphite  nozzle  on  a 
space  booster  engine  required  to  increase  performance.  Figure  3 
provides  a  sample  list  of  parameters  for  a  number  of  weapon 
systems.  Parameters  selected  must  be  measurable  if  they  are 
expected  to  provide  technical  progress. 

The  selection  of  technical  parameters  begins  in  the  demon¬ 
stration/validation  phase  with  the  recommendations  of  the 
participating  contractor ( s )  system  engineers.  The  final  selec¬ 
tion  of  the  parameters  must  be  made  by  the  PM.  The  selected  and 
contractually  implemented  technical  parameters  will  be  the 
product  of  detailed  negotiations  between  the  prime  FSD  contrac¬ 
tors)  and  the  government  PM.  The  PM's  decision  will  be  based 
on  the  operating  commands  needs,  engineering  analysis  of  risk, 
OSD  desires  and  the  contractor's  guidance. 

The  PMO  cost  of  TPM  is  proportional  to  the  total  number  of 
parameters  which  must  be  tracked.  To  report  technical  variance 
at  the  system  level  requires  the  contractor  to  track  and  report 
at  the  subsystem  level  and  aggregate  the  data.  The  tracking  of 
10  or  20  key  system  level  parameters  may  result  in  the  contrac¬ 
tor  being  required  to  track  several  hundred  lower  level  com¬ 
ponent  parameters.  Figure  4  displays  the  flow-down  used  by  an 
engine  manufacturer  to  track  system  reliability  (Mean  Time 
Between  Failure  [ MTBF ] ) .  To  permit  determination  of  the  cause 
of  unfavorable  variance  predictions,  the  contractor  tracked 
performance  on  11  subsystems  and  13  unique  components  as  well  as 
the  total  engine. 

The  parameters  ultimately  selected  for  tracking  and  report¬ 
ing  should  satisfy  the  following  criteria: 

o  Each  parameter  tracked  or  reported  should  be  correlated 
with  a  specific  CWBS  element. 


AIRCRAFT 


SHIP 


WEIGHT  -  MAX  TAXI,  EMPTY,  MAX  FLT  CRUISING  RANGE 
PAYLOAD  -  INTERNAL,  EXTERNAL  MAX  SPEED 

RANGE  DIMENSIONS  -  LENGTH,  BEAM 

SPEED  -  HIGH,  LOW,  PENETRATION  DRAFT 

LANDING  DISTANCE  DISPLACEMENT 

RADAR  CROSS  SECTION*  TARGET  ACQUISITION  RANGE 

TURNAROUND  TIME  RELOAD  TIME* 

MAINTAINABILITY  NO.  TARGETS  ENGAGED 

MISSION  RELIABILITY  SIMULTANEOUSLY 

TANK 

DIMENSIONS  -  HEIGHT,  WEIGHT,  WIDTH 
SPEED  -  MAX,  GRADE,  ACCELERATION 
CRUISING  RANGE 
SURVIVABILITY* 

MAINTAINABILITY 
MOBILITY 
FORDABILITY 

*Could  be  state  of  the  art  critical  if  new  technology  required. 


SATELLITE 

DIMENSIONS  -  WEIGHT,  SIZE 
POWER* 

RELIABILITY 

DRIFT* 

COMPUTER  LOADING 
TELEMETRY  ALLOCATION 


Figure  3.  TPM  Parameters 


Figure  4.  Parameter  Flow  Down 


o  A  parameter  time-phased  profile  with  tolerance  bands  can 
be  predicted  and  substantiated  during  design,  develop¬ 
ment  and  test. 

o  A  direct  measure  of  value  can  be  derived  from  results  of 
functional  analysis. or  tests. 

o  Will  have  a  significant  effect  on  system  ability  to  meet 
specified  performance  requirements. 

Subjective  items  such  as  improved  quality,  management  respon¬ 
siveness  and  timeliness  are  not  appropriate  for  TPM. 

2.3.5  Parameter  Profile  Determination 

In  the  May  1982  issue  of  Research  Management,  Mr.  Alfred  H. 
Schainblatt  concluded  that  the  "idea  of  measuring  R&D  produc¬ 
tivity  makes  sense  only  if  there  are  reasonable  comparisons.* 
Similarly,  the  measurement  of  technical  progress  makes  sense 
only  if  reasonable  schedules  can  be  established.  Realistic 
performance  profiles  for  evaluating  current  progress  and  fore¬ 
casting  levels  of  attainment  must  be  carefully  set  to  prevent 
early/late  or  unnecessary  management  intervention;  any  of  which 
would  mean  that  the  cost  of  TPM  would  have  been  wasted  at  best 
and  counter  productive  at  worst. 

A  profile  can  be  defined  as  a  graphic  depiction  of  a  pro¬ 
cess  or  relationship  which  serves  the  following  functions: 

o  provides  a  visual  presentation  of  present  status  and 
expected  accomplishment 

o  provides  a  means  of  presenting  predictions  and  planned 
corrective  action  effects 

o  relates  time,  performance  and  significant  events  both 
past  and  present 
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The  shape  of  this  profile  will  determine  the  overall  effec¬ 
tiveness  of  a  TPM  program.  A  profile  that  is  too  optimistic 
(e.g.,  early  attainment,)  will  increase,  the  probability  of 
performance  variances  even  though  actual  progress  may  satisfy 
the  overall  program  schedules.  Conversely,  a  conservative 
profile  may  not  identify  problems  until  it  is  too  late  to 
effectively  accomplish  corrective  actions.  Therefore,  the 
parameter  profiles  must  represent  the  reasonable  expectations 
of  technical  progress  based  on  the  experience  of  prior  similar 
design  activities  and  current  program  design  completion,  proto¬ 
type  assembly,  and  component  test  schedules. 

Some  performance  characteristic  predictions  may  tend  upward 
or  downward  with  time,  such  as  engine  reliability  ( MTBF ) , 
whereas  others  may  appear  as  a  horizontal  line  if  it  is  reason¬ 
able  to  forecast  that  the  parameter  will  be  constant  (electri¬ 
cal  power  consumption).  Figure  5  provides  planned  parameter 
profiles.  Figure  6  displays  a  planned  profile  with  an  upper 
and  lower  tolerance  band.  The  intent  is  that  performance 
outside  the  tolerance  band  (both  positive  and  negative)  would 
result  in  variance  reporting.  In  this  case  the  demonstrated 
performance  is  outside  the  tolerance  band  and  variance  report¬ 
ing  to  include  planned  corrective  action  would  be  required. 
Performance  profiles  may  also  be  developed  in  a  tabular  form, 
comparing  demonstrated  and  forecast  values  to  planned  value  and 
providing  variance.  Figure  7  is  a  tabular  TPM  comparison. 

2.3.6  Parameter  Measurement 

The  method!  used  to  measure  technical  progress  must  be 
tailored  to  the  particular  phase  of  the  development  program. 
In  the  initial  phases  of  design,  only  engineering  analysis  is 
available.  As  the  design  matures,  the  analysis  is  augmented 
with  engineering  hardware  test  data.  Finally,  component  and 
full-system-test  hardware  allows  for  performance  verification. 
TPM  parameter  measurements  are  basically  the  specialized  docu- 


PARAMETER  PARAMETER 


Figure  6.  TPM  PARAMETER  PROFILE 
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Figure  7.  TPM  Ccnparison 
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mentation  of  the  natural  outputs  from  the  ongoing  development 
activities  involving  engineering  analysis  and  testing  (Figure 
1).  TPM  should  not  routinely  have  to  introduce  testing  or 
analysis  events  not  otherwise  required  in  every  design  pro¬ 
ject.  The  system  engineering  plan  should  structure  the  techni¬ 
cal  design  effort  so  that  it  naturally  provides  the  events  and 
measurement  activities  which  TPM  reporting  will  require. 


2. 3. 6.1  Engineering  Analysis 


Throughout  the  development  activity  many  techniques  are 
used  to  identify  and  evaluate  design  options.  This  evaluation 
is  termed  Engineering  Analysis,  and  incorporates  activities 
such  as  modeling,  comparisons/similarity/experience  determina¬ 
tions,  and  formal  trade  studies. 

MODELING  (Mathematical): 

Static  math  modeling  provides  equations  which  dupli¬ 
cate  the  known  relationships  between  characteristics 
when  a  system/subsystem/component  is  in  equilibrium. 
This  technique  can  be  used  in  establishing  quantitative 
performance  requirements  for  the  development  of  candi¬ 
date  designs.  Structural  loads  analysis  of  a  design  is 
an  example  of  static  math  modeling  with  widely  demon¬ 
strated  validity  and  ready  adaptation  to  automation. 
Dynamic  math  models  describe  conditions  that  vary  with 
time.  Simple  dynamic  relationships  can  be  solved  manu¬ 
ally  and  the  results  plotted.  However,  in  modern  weapon 
systems,  the  degree  of  system  complexity  requires  com¬ 
puter  simulation,  which  also  provides  increased  designer 
productivity,  more  rapid  response  times,  and  reduced 
test  hardware  needs. 

MODELING  (Physical): 

Physical  models  include  both  scale  models  and  full 
size  mock-ups.  Scale  models  may  be  used  to  establish 


specific  characteristics  for  subsequent  use  in  computer 
simulations  and  model  tests.  As  examples,  scale  models 
of  aircraft  are  used  in  wind  tunnels  to  establish  lift 
and  drag  characteristics,  while  scale  models  of  ship 
hulls  are  used  in  water  tanks  to  determine  drag  resis¬ 
tance  and  acceleration  characteristics.  Full  size 
models  are  usually  constructed  to  support  human  engi¬ 
neering  functions,  and  those  for  which  a  three  dimen¬ 
sional  presentation  will  aid  in  the  design  of  tubbing/ 
cable  routings,  component  placement,  and  the  sub¬ 
sequent  accessibility  analysis.  The  implementation  of 
Computer  Aided  Design  (CAD)  Mock-Ups  will  reduce  the 
need  for  physical  modeling. 

Dynamic  models  are  primarily  engineering  hardware 
models  used  to  provide  proof  of  functional  operation  or 
to  establish  critical  performance  characteristics. 
Breadboard/brassboard  of  an  electronic  circuit  is  an 
example  of  a  dynamic  model.  The  model  did  not  physi¬ 
cally  look  like  the  anticipated  operational  configura¬ 
tion,  but  functionally  it  was  intended  to  be  very  repre¬ 
sentative.  In  some  cases  CAD  can  now  accomplish  the 
basic  circuit  design  and  functional  tests  without  the 
need  for  physical  breadboards.  Data  obtained  from  this 
type  of  engineering  model  or  hardware,  which  function¬ 
ally  approximates  the  desired  design,  allows  for  con¬ 
firmation  of  approach  and  degree  of  design  optimization. 

COMPARISON/SIMILARITY/EXPERIENCE 

These  techniques  are  predominant  in  the  early  phases 
of  the  engineering  process.  First  order  design  approxi¬ 
mations  will  draw  heavily  on  relationships  developed  in 
previous  similar  programs.  As  an  example,  in  making  an 
initial  estimate  of  aircraft  weight,  the  engineer  would 
evaluate  earlier  aircraft  designs  for  a  similar  mission 
and  develop  a  weight  relationship.  Using  this  relation¬ 
ship  and  the  geometry  of  the  new  aircraft,  an  estimate 


of  total  weight  is  developed.  This  estimate  may  then  be 
adjusted  based  on  an  assessment  of  the  evolving  tech¬ 
nology  in  aircraft  structures.  In  developing  the  elec¬ 
trical  power  requirement  for  a  new  ship,  design  engi¬ 
neers  would  evaluate  similarly  equipped  ships  and  devel¬ 
op  a  power  relationship  based  on  the  power  requirements 
for  similar  equipment. 

These  techniques  are  useful  in  developing  the  re¬ 
source  requirement  estimates  for  the  engineering  ef¬ 
fort.  The  establishment  of  development  schedules, 
program  person  loading,  and  parameter  time  profiles  are 
highly  dependent  on  prior  engineering  experience  in 
similar  activities.  Although  these  techniques  are 
useful  in  the  concept  exploration  and  early  demonstra¬ 
tion/validation  phases,  they  are  the  least  preferred 
activities  for  measuring  technical  progress  because  of 
their  subjectivity  and  the  difficulty  in  independently 
verifying  and  validating  these  results.  While  some  TPM 
estimates  may  initially  depend  on  these  methods,  their 
refinement  by  other  methods  should  be  a  primary  goal  of 
the  TPM  program  at  start  up. 

TRADE  STUDIES; 

Trade  studies  formally  apply  elements  of  decision 
theory  and  multi-attribute  utility  functions  to  select  a 
design-alternative  that  best  satisfies  a  selected  set  of 
decision  criteria.  During  Full  Scale  Development  (FSD), 
when  formal  TPM  is  in  effect,  trade  studies  are  used  in 
detailed  design  analysis  to  identify  the  most  desirable 
design  alternatives  considering  design  criteria  selected 
by  the  system  engineering  manager  such  as  reliability, 
weight,  cost,  speed,  size,  etc.  For  the  M1E1  tank 
system,  the  SEMP  required  that  each  trade  study  consider 


the  following  factors,  as  appropriate,  and  their  respec¬ 
tive  impacts: 


o  Performance 
o  Maintainability 
o  Human  Factors/Safety 
o  Development  Cost 
o  Life  Cycle  Cost 


o  Reliability 
o  Durability 

o  Integrated  Logistics  Support 
o  Production  Cost 
o  Schedule 


The  structured  trade-off  analysis  procedure,  when  made 
mandatory,  prevents  the  premature  commitment  to  a  single 
design  prior  to  evaluating  all  viable  alternatives.  It 
requires  close  management  attention  since  it  costs  time 
and  money  to  implement  and  operate  such  a  disciplined 
system. 


The  process  consists  of  evaluating  all  feasible 
solutions  against  selected  criteria  which  have  been 
prioritized  by  weighting.  Specific  measured/predicted 
performance  information  is  developed  by  the  use  of 
previously  mentioned  engineering  analysis  techniques. 
The  performance  of  each  alternative  against  the  criteria 
is  scored  based  on  the  predicted  level  of  attainment. 
To  provide  consistency  in  scoring,  utility  function 
curves  are  developed,  prior  to  performance  determination 
which  represents  the  score  for  varying  levels  of  per¬ 
formance  of  each  attribute.  The  candidates  are  then 
ranked  based  on  the  weighted  scores  developed  from  the 
summation  of  the  individual  raw  scores  multiplied  by  the 
weighting  factors  which  have  been  separately  designed  to 
reflect  desired  criteria  priority  (Figure  8). 


Although  this  process  yields  a  quantitative  rating  of 
candidate  systems/solutions,  major  segments  of  the 
process  are  subjective  in  nature.  The  rank  ordering  and 


CANDIDATE 

SYSTEMS 

DESIGNATED 


relative  weighting  of  the  criteria  is  a  function  of 
management's  perception  of  the  importance  of  these 
factors.  Depending  on  the  particular  program,  the 
acceptability  of  risk,  the  fiscal  or  political  consid¬ 
eration,  or  the  personnel  ceilings  may  take  precedence 
at  any  given  time.  Trade  study  outputs  should  always  be 
subjected  to  sensitivity  analysis  to  determine  the  range 
of  conditions  which  would  still  yield  the  same  results. 
If,  the  range  of  conditions  for  which  the  outcome  remains 
unchanged  is  relatively  broad  and  encompasses  the  per¬ 
ceivable/predictable  conditions  for  the  system,  the 
trade  study  results  should  be  utilized.  If  the  range  of 
conditions  is  very  narrow  and  there  is  a  probability  of 
result  reversal  -  additional  analysis  should  be  con¬ 
ducted,  time,  personnel  and  funds  permitting,  or  the 
options  should  remain  open  until  additional,  more  defin¬ 
itive  information  can  be  obtained.  Figure  9  provides 
the  results  of  a  typical  trade  study. 

2. 3. 6. 2  Testing 

It  is  DoD  Policy  (DoDD  5000.3)  that  Test  and  Evaluation 
( T&E )  begin  as  early  as  possible  and  be  conducted  throughout 
the  system  acquisition  process.  Planned  T&E  activity  should 
assess  and  reduce  acquisition  risks  and,  as  soon  as  possible, 
allow  for  the  estimation  of  operational  effectiveness  and 
suitability  of  the  system  under  development.  DoD  further 
requires  that  meaningful  test  objectives  and  evaluation  crite¬ 
ria,  related  to  the  satisfaction  of  mission  need,  be  estab¬ 
lished  before  testing  begins.  Successful  accomplishment  of 
these  T&E  objectives  will  be  instrumental  in  obtaining  deci¬ 
sions  to  commit  additional  resources  to  a  program  or  to  advance 
it  from  one  acquisition  phase  to  another. 

Maximum  test  activity  occurs  during  the  FSC  phase  of  the 
acquisition  cycle,  the  same  period  for  which  TPM  is  formally 
instituted.  Testing  is  a  major  function  of  TPM.  It  provides 
the  validation  of  the  engineering  analysis  previously  completed. 
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PRESENT  DESIGN:  S1451  Cooling  System 


XM61462  Electro-Mechanical  Clutch  (Right  auxiliary  cooling  fan) 
XM69078  Mechanical  Clutch  (Left  primary  cooling  tan) 


PROPOSED  DESIGN: 

XM69078  Two  Mechanical  Clutch  Designs 
(Identical  right  and  left  cooling  fan  slip  clutches) 


PARAMETER 


Performance 


Reliability 


Maintainability 


Durability 


Human  Factors 


IMPACTS 


Present  Design  Proposed  Design 


Speed  107  Slope  -  20.17  MPH 
Acceleration 

(0  to  MPH)  -  6.49  sec 


15  x  10  -6  Failures/Mile  i.e. 
105.97  Vehicles  Failures/Year 


2.29M-HRS  (Remove  Replace) 


Usage  rate  6000  miles 
Overhauls  3  times  over 
20  years 


Installation  and  Maintenance  as 
specified  in  MIL-STD-1472A 


Provision  Requirements  for 
Electro-Mechanical  and 
Mechanical  Clutches 


19.63  MPH  (-2.77  Change) 
6.69  sec  (-2.97  Change) 


Same 


Weight 


Improved  Logistics 
Identical  Right  and  Left 


-30  lbs. 


Overall  Assessment  -  Technical  Impacts: 

Minor  degradation  in  tank  performance  and  a  minor  degradation  in  Fuel  Economy. 
Electrical  Clutch  -  0.58  mpg.  Mechanical  Clutch  -  0.57  mpg  (-1.57) 


$468.00  per  unit 


$5,075 


$10,497 


$2,428  system  cost 


SEM  Analysis  and  recommendations: 

Advantages  of  using  the  slip  clutch  outweigh  minor  degradation  in  tank 
performance.  An  LCC  fleet  saving  of  $7.3  million  is  achieved  for  20  years 
vehicles  operating  period. 


Trade/Risk  Board  disposition: 

Change  approved  by  GDLS  Submitted  to  the  Government  for  implementation 
through  ECP. 


DTC 

$731.81  per  unit 

Development 

Cost 

$93,348 

Non-recurring 

Cost 

$16,854 

LCC 

$3,467  system  cost 

Schedule  Considerations: 

PC-3  Production,  Retrofit  FV-2  and  FV-3 

Figure  9.  Trade  Study  Summary  -  Example 


There  are  two  types  of  tests,  functional  and  environ¬ 
mental.  Functional  tests  are  used  to  determine  if  the  perform¬ 
ance  of  the  item  meets  its  functional  ,  performance  require¬ 
ments.  These  tests  are  either  electrical  or  mechanical.  An 

electrical  functional  test  is  the  measurement  of  output  power 
from  a  radar  power  unit  while  a  mechanical  functional  test 

could  be  torque  measurement  of  the  antenna  rotation. 

Environmental  tests  are  those  tests  which  subject  the  item 
to  the  environmental  conditions  in  which  its  functional  per¬ 

formance  will  occur.  They  include  the  environments  of  shock, 
vibration,  acceleration,  radiation,  temperature,  electro¬ 
magnetic  compatibility,  etc. 

Because  of  the  multitude  of  agencies  involved  in  the  vari¬ 
ous  phases  of  testing  and  the  many  applications  for  the  resul¬ 
tant  data,  it  is  imperative  that  careful  planning  of  all  phases 
(including  data  distribution)  be  accomplished.  The  Test  and 
Evaluation  Master  Plan  (TEMP)  should  document  this  planning  by 
clearly  showing  that  the  various  phases  of  Development,  Test 
and  Evaluation  ( DT&E )  and  Operational  Test  and  Evaluation 

( OT&E )  are  structured  to  ensure  that  critical  management  is¬ 
sues,  at  each  decision  point,  can  be  satisfied  with  verified 
performance  data.  The  TEMP  is  prepared  by  the  government  PM's 
test  organization  and  serves  as  the  basis  for  all  contractor 
test  plans.  The  TEMP  should  contain  six  parts.  They  are  as 
follows : 

o  Part  I  Description 

Mission  -  brief  summary  of  operational  need,  mission  and 
planned  operational  need 

System  -  key  functions,  interfaces  and  unique  charac¬ 
teristics. 

Required  Operational  Characteristics 
Required  Technical  Characteristics 
Critical  T&E  Issues 


Part  II  Program  Summary 


Management  -  how  the  tests  will  be  managed 
Integrated  Schedule 

o  Part  III  DT&E  Outline 

Relate  test  objectives  to  system  operational  concept 
DT&E  to  date 

Future  DT&E  to  include  equipment,  objectives,  events  and 
critical  items 

o  Part  IV  OT&E  Outline 

Relate  the  test  conditions  and  results  to  operational 
effectiveness  and  suitability 

test  representative  of  operational  environment 

o  Part  V  Production  Acceptance  Test  and  Evaluation  ( PAT&E ) 

Plan  to  demonstrate  that  items  fulfill  requirements  and 
specifications 

o  Part  VI  Special  Resource  Summary 

-  Key  resources  for  DT&E,  OT&E  and  PAT&E 
Test  articles 

Special  support  requirements 
DEVELOPMENT  TEST  and  EVALUATION 

DT&E  is  conducted  to  assist  the  engineering  design  and 
development  process  and  to  verify  attainment  of  technical 
performance  specifications  and  objectives.  It  encompasses  the 


T&E  of  components,  subsystems,  hardware/software  integration, 
and  prototype  or  full  scale  development  models  of  the  system. 
It  should  ensure  that: 

o  engineering  design  and  development  are  reasonably  com¬ 
plete 

o  all  significant  design  problems  have  been  identified 
o  solutions  to  identified  problems  have  been  developed 
o  design  risks  have  been  minimized 
o  the  system/item  will  meet  its  specifications. 

DT&E  for  the  M-l  tank  program  culminated  with  the  competition 
of  prototypes  developed  by  General  Motors  and  Chrysler,  de¬ 
signed  to  prove  the  above  noted  functions,  but  began  with  bench 
tests  of  single  bearing  and  switch  designs. 

The  management  of  the  DT&E  activities  is  usually  shared 
between  the  contractor  and  the  government.  The  contractor 
normally  conducts  the  tests  at  his  own  facilities.  The  govern¬ 
ment  may  support  some  tests  by  providing  specialized  test 
facilities,  equipment  and  personnel,  such  as  wind  tunnels  and 
flight  test  ranges. 

OPERATIONAL  TEST  and  EVALUATION 

This  is  the  T&E  activity  conducted  to  estimate  a  system's 
operational  effectiveness  and  suitability,  and  to  provide 
information  on  tactics,  doctrine,  organization,  and  personnel 
requirements.  It  is  DoD  policy  that  an  initial  phase  of  this 
testing  (IOT&E)  be  conducted  prior  to  the  production  decision. 

Public  Law  98-94,  dated  1  NOV  83,  established  a  separate 
organization,  the  Directorate  of  Operational  Test  and  Evalua¬ 
tion,  which  reports  directly  to  the  Secretary  of  Defense  to 
oversee  OT&E  on  major  system  acquisitions.  Based  on  the  re¬ 
ports  of  the  independent  testing  organizations  and  personal 
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visits  and  analysis,  the  Director  reports  to  Congress  on  the 
operational  effectiveness  and  suitability  of  major  system 
acquisitions.  The  reports  are  forwarded  without  change  to 
Congress  through  the  Secretary  of  Defense. 

OT&E  testing  differs  from  DT&E  in  that  it  is  accomplished 
in  an  environment  "as  operationally  realistic  as  possible". 
Government  operational  and  support  personnel  are  used  in  lieu 
of  contractor  personnel  to  obtain  valid  estimates  of  the  users' 
capability  to  operate  and  maintain  the  system.  Contractor 
personnel  might  be  used  to  assist  in  data  gathering,  reduction 
and  evaluation. 

OT&E  is  conducted  to: 

o  estimate  the  system's/item's  military  utility,  opera¬ 
tional  effectiveness  and  suitability 

o  assess  the  compatibility,  interoperability,  safety, 
reliability,  maintainability,  and  supportability 

o  provide  data  to  support  or  assess  the  validity  of  opera¬ 
tional  instructions,  publications  and  handbooks 

o  assess  the  procedures  and  equipment  planned  for  the 
support  of  the  fully  fielded  material. 

2.4  DOCUMENTATION 

A  major  effort  in  any  development  program  is  the  production 
and  delivery  of  documentation.  The  Data  Item  deliveries  as 
required  by  the  Contract  Data  Requirements  List  { CDRL )  can 
account  for  three  to  five  percent  of  the  total  cost  associated 
with  the  development  program.  Engineering  plans,  test  plans, 
analysis  reports,  test  reports,  cost  and  management  status 
reports,  plus  many  other  types  of  specialized  functional  data 
comprise  the  program  details  of  what  is  to  be  done  and  how  well 
it  is  being  accomplished.  Within  this  myriad  of  standard  data 
items  there  are  several  that  are  key  to  cost  effective  TPM. 
The  following  subsections  discuss  the  requirements  of  the  Data 
Item  Descriptions  (DID)  which  may  be  placed  on  contract  to 
implement  the  TPM  program. 


2.4.1  System  Engineering  Management  Plan  (SEMP)  DI-S-3618/ 


S-152 )  ( USAF ) 

The  SEMP  sets  forth  the  contractor's  proposed  plan  for 
conduct  and  management  of  a  fully  integrated  engineering  effort 
(including  internal  procedures)  necessary  to  satisfy  the  gen¬ 
eral  and  specific  requirements  of  MIL-STD-499A  as  implemented 
by  the  contract  schedule  or  Statement  Of  Work  (SOW).  A  con¬ 
tractor's  SEMP  is  usually  submitted  as  part  of  a  response  to  a 
request  for  proposal  in  which  system  engineering  is  specified 
in  the  SOW.  The  SEMP  is  divided  into  three  parts: 


Part  1,  System  Engineering  contains  a  description  of  the 
contractors  system  engineering  process  as  proposed  for  applica¬ 
tion  to  the  definition  of  system  design  and  test  requirements. 
It  will  contain  the  contractor's  proposed  plans,  processes  and 
procedures  for: 


o  Functional  analysis 
o  Requirements  allocation 
o  Trade  studies 

o  Design  optimization/effectiveness  analysis 
o  Synthesis 

o  Technical  interface  compatibility 
o  Logistics  support  analysis 
o  Productibility  analysis 
o  Generation  of  specifications 

Part  2,  Technical  Program  Integration  contains  the  contrac¬ 
tor's  proposed  technical  program  planning  and  control  of  his 
engineering  efforts  for  design,  development,  test  and  evalua¬ 
tion  functions.  It  shall  provide  the  plan  for: 

o  Risk  analysis 
o  Engineering  integration 
o  Contract  Work  Breakdown  Structure 
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o  Assignment  of  responsibility  and  authority 
o  Program  reviews 

o  Technical  Performance  Measurement 
o  Interface  control 
o  Documentation  control 

Part  3,  Engineering  Integration  contains  the  engineering 
specialty  programs  proposed  in  accordance  with  applicable 
military  standards.  It  will  include  methods  by  which  the 
contractor  proposes  to  integrate  these  specialty  programs.  In 
addition,  this  part  will  include  a  summary  of  each  specialty 
program  such  as  reliability,  maintainability,  safety,  surviv¬ 
ability,  vulnerability,  electromagnetic  compatibility,  etc 
Figure  10. 


2.4.2  Technical  Performance  Measurement  Report  (DI-S-3619/ 
S-153)  ( USAF ) 


The  Technical  Performance  Measurement  Report  is  designed  to 
provide  visibility  to  the  program  manager  on  the  state  of 
engineering  accomplishment  toward  the  contract  requirements 
compared  to  planned  and  required  values."  The  reportable  Cl 
elements  and  parameters  to  be  included  in  this  report  will  be 
those  listed  in  the  SEMP.  For  each  parameter  selected  for  TPM 
reporting,  reports  shall  include: 


o  "The  demonstrated  value,  planned  value,  and  demonstrated 
variance  for  the  design  at  the  time  of  TPM,  plus  the 
current  estimate,  the  current  specification  requirement 
and  the  predicted  variance  for  the  end  product." 

o  "Configuration  design  status  and  discussion  of  design 

and  engineering  investigations  and  analyses  *‘-sich  sup¬ 
port  the  demonstrated  value,  and  discussion  of  the 

technical  effort  which  supports  the  predicted  profile 

leading  to  the  current  estimate." 

o  "Variance  Analysis  to  include  discussion  of  design, 

development,  and/or  fabrication  problems  encountered 
which  cause  demonstrated  or  predicated  performance 
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DATA  ITEM  DESCRIPTION 

I.  TITLE 

System  Engineering  Management  Plan  (SEMP) 

a  OLSC  *i*T  10*4  '  PUH+OiK 


g _ IDENTIFICATION  wots* 


AGENCY 

NUMBER 

DI-S-3618/ 

S— 152 

4.  APPROVAL  OATS 


To  set  forth  the  contractor's  proposed  plan  for  the  conduct 
and  management  of  the  fully  integrated  engineering  effort 
necessary  to  satisfy  the  general  and  specific  requirements 
of  M1L-STD-499  as  implemented  by  the  contract  schedule  or 
statement  of  work. 


9  Feb  1970 

T  OFFICE  or  PRM..HT 
reeronairilF 

AFSC 

•  OOC  REQUIRED 


•  APPROVAL  LIMITATION 


T  APPLICATION/ INTERREL  ATIONAMIP 


Acquired  as  a  product  of  the  Contract  Definition  Phase  IB 
(or  equivalent),  this  plan  is  used  in  the  evaluation  of  a 

contractor's  proposal.  This  plan  will  be  maintained  current  »  references  (tfndm'ory « THUS  m 
and  may  become  a  part  of  the  acquisition  contract  statement 
of  work. 

MIL- STD-4 99 


MC*L  NUMtERIB 


10  PREPARATION  INITRUCTIONI 

The  SEMP  will  be  prepared  in  accordance  with  the  applicable  paragraphs  of  the  require¬ 
ments  section  of  MIL-STD-499,  System  Engineering  Management.  This  plan  shall  set  forth 
a  description  of  the  contractor's  proposed  efforts  for  the  planning,  control,  and 
conduct  of  a  fully  integrated  engineering  effort  to  satisfy  general  and  specific 
requirements  of  MIL-STD-499  as  tailored  by  the  contract  statement  of  work.  The 
SEMP  shall  contain  sufficient  detail  to  establish  that  the  contractor's  proposed 
process  (including  internal  procedures),  his  management,  and  the  extent  of  the  planned 
application  of  the  process,  will  satisfy  the  requirements  of  the  Standard  as  tailored 
by  the  statement  of  work.  The  SEMP  shall  include  any  contractor  recommendations  for 
further  tailoring  of  the  standard  to  the  particular  contractual  effort.  Those  portions 
of  the  SEMP  proposed  by  the  contractor  to  become  contractual  requirements  shall  be 
denoted.  This  plan  shall  be  divided  into  three  parts: 

Part  1  -  System  Engineering. 

Part  2  -  Technical  Program  Planning  and  Control. 

Part  3  -  Engineering  Specialty  Integration. 


DD- 522“..  1664 
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Figure  10.  SYSTEM  ENGINEERING  MANAGEMENT  PLAN 
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outside  the  planned  tolerance  band.  When  this  occurs,  a 
revised  planned  value  profile  will  be  presented.  The 
contractor  will  report  impacts  on  higher  level  parame¬ 
ters,  on  interface  requirements  and  on  system  cost 
effectiveness  if  appropriate.  For  performance  deficien¬ 
cies  alternate  and  proposed  recovery  plans  and  associ¬ 
ated  configuration  changes  will  be  reported  with  the 
performance,  cost,  and  schedule  implications  of  each." 

o  Appropriate  graphical  documentation  of  results  (Figure 

11). 

2.4.3  Engineering  Management  Report  (Dl-M-30417) (USAF) 

This  report  provides  the  PMO  with  current  status  informa¬ 
tion  and  an  alert  to  significant  program  problems.  Section  One 
provides  a  summary  of  overall  progress,  either  favorable  or 
unfavorable.  Section  Two  requires  a  report  of  each  major 
technical  area  under  current  analysis,  its  technical  progress, 
technical  problems,  alternative  plans  and  significant  meetings 
(Figure  12). 

2.4.4  Subsystem/Engineering  Development  Report  (DI-S-3582/ 
S-102-1)  (USAF) 

This  report  provides  "...  visibility  to  the  program/ 
project  manager  on  the  state  of  engineering  development  of 
Cl/subsystem,  as  weil  as  providing  a  basis  for  projecting 
needed  supporting  efforts  such  as  verification  testing  and 
production  scheduling...  These  data  are  related  to  the  design 
requirements  for  the  Cl/subsystem  and  to  the  design  analyses 
conducted  on  the  item  during  the  design  process...  This  report 
is  ...  applicable  to  other  efforts  where  information  on  design 
evolution  is  necessary  to  support  program  decisions  and  related 
developmental  efforts.  This  report  will  describe  ...  such 
information  as:  predicted  performance  compared  to  and  based 
upon  the  design  requirements  characteristics,  discussion  of 
design  and  development  investigations,  and  design/development 
and/or  fabrication  problems  encountered,  with  proposed  solu¬ 
tions  and  probable  impact*  (Figure  13). 
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DATA  ITEM  DESCRIPTION 


IM«T|»IC>T19«  BCXII 


Technical  Performance  Measurement  Report 


US  AT  ls-153 


It  MKRiRTiOft/RwRRDII 


To  provide  visibility  to  the  progm/projeet  manager  on  the 
state  of  engineering  accomplishment  toward  the  contract 
requirements  as  compared  with  planned  and  required  values. 
Provides  a  basis  for  projecting  needed  supporting  efforts. 


«-  VPMfU.  SlTI 

9  Peb  1970 

t.  STfTct  e,  Ti-ttaa* 


Is.  eoc  utgiau 


I.  **PmO**L  UMTATlDH 


IT.  AD»VlCATtt«/IM 


These  data  are  related  to  the  design  requirements  (Part  X 
of  the  specification)  for  the  Configuration  Itea  (Cl)  and 
to  its  critical  elements  and  design  parameters.  The 
reportable  Cl  elements  and  parameters  to  be  included  in 
this  report  trill  be  those  listed  in  the  System  Engineering 
Management  Plan  Incorporated  as  a  contract  requirement  and 
trill  be  identified  on  the  DD  Fora  1423  either  by  attachment 
or  reference  to  the  SIMP.  This  DID  will  normally  be  used 
only  when  MIL-STD-499  is  a  contractual  requirement;  other** 
vise,  DD  Form  1664,  DI-S-3582  or  5-102,  should  be  used. 
Should  this  DID  be  preferred  to  DI-S-3582  or  S-102,  when 
Ml L-STD-499  is  not  a  contractual  requirement,  the  task 
effort  for  generating  this  data  must  be  included  in  the 
contract  work  statement. 


MIL-STD-499 
AFR  375-7 
AFS04/4FL«  310-1 


I  aCk.  ttuHtlMl 


I  m  nmiatTM  iKtauciiui 


1.  The  contractor  shall  prepare  a  TPM  report (s)  on  designated  parameters.  The  DD 
Form  1423  will  specify  whether  a  particular  report  will  cover  all  parameters  of  a 
system  element,  an  Individual  parameter,  or  selected  groupings  of  parameters. 

2.  For  each  parameter  selected  for  TPM  reporting,  reports  shall  Include: 

a.  The  demonstrated  value,  planned  value,  and  demonstrated  variance  for  the  design 
at  the  time  of  the  TPM.  plus  the  current  estimate,  the  current  specification  require¬ 
ment  and  the  predicted  variance  for  the  end  product.  Determination  of  the  current 
estimate  shall  be  based  on  the  demonstrated  value  and  the  changes  to  the  parameter 
value  which  can  be  attained  vlthin  the  remaining  schedule  and  cost  baseline.  The 
format  shall  be  as  described  in  paragraph  3  below. 

b.  Status  of  the  configuration  design  and  discussion  of  design  and  engineering 
Investigations  (e.g.,  experiments  and  tests  performed)  and  analyses  which  support  the 
demonstrated  value,  and  discusaion  of  the  technical  effort  which  supports  the  predicted 
profile  leading  to  the  current  estimate. 

c.  Variance  Analysis  to  include  discussions  of  design,  development,  and/or 
fabrication  problems  encountered  which  cause  demonstrated  or  predicted  performance 
outside  the  planned  tolerance  band.  Uhen  this  occurs,  a  revised  planned  value  profile 
will  be  presented  as  shown  in  Figure  2.  The  contractor  will  report  impacts  on  higher 
level  parameters,  on  interface  requirements  and  on  system  cost  effectiveness  if 
appropriate.  For  performance  deficiencies,  alternate  and  proposed  recovery  plans 
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Figure  11.  TPM  REPORT 
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DATA  ITEM  DESCRIPTIOM 

I.  Tlttl 

Engineering  Management  Report 
•  DUfiAi7r!e7»MMo!i  ' 


i* 


AGCNCV 

MUMftCJt 

USAF 

DI-M-3M17 

4.  >P>UV4k  DATK 


The  purpose  of  this  report  is  to  keep  top  progrea  manage¬ 
ment  Informed  of  vhat  la  happening,  give  them  warning 
signals  of  impending  trouble  and  to  direct  their  attention 
to  significant  program  problems. 


1  Mar  1970 

•  erf.citr.iwuir 
•  CI»ONIllH.fYv 

APSC 


*  odc  ncouii 


j*.  A^MklCATIOK/l 


IONIMIP 


Broadly  speaking,  the  report  should  give  the  answer  to  the 
question,  "How  are  we  doing!"  It  is  intended  that  this 
report  will  be  distributed  at  levels  equal  to  and  above 
the  SFD. 


•  BtriMCNCCSf^ndalaryM  c/ia<M 
fcl*cJt  lOj 


MCll  NUM»K»cai 


Formerly  UM-672  (ASD) 

I*.  ihitmuctioh* 

Section  I 


A.  Provide  a  two  page  summary  pertaining  to  the  overall  progress  (favorable 
and  unfavorable)  of  the  program. 


Section  II 

A.  The  outline  of  the  report  shall  be  applied  to  each  major  technical  area 
under  current  analysis.  Each  area  will  be  discussed  separately  in  terms  of: 

1.  Technical  Progress 
(favorable  and  unfavorable) 

2.  Technical  Problems 

In  discussing  a  specific  problem,  it  is  expected  that  the  contractor 
will  identify  and  ilacuaa  the  root  cauae  to  whatever  level  is  necessary. 

3.  Plan  for 

(Several  alternatives  and  their  related  advantages  and  disadvantages 
should  be  discussed  that  lead  to  the  specific  course  of  action  decided  upon. 

4.  Significant  Meetings 


OATA  ITEM  DESCRIPTION 

AGENCY 

NUMGCR 

».  title 

Subsystem  /Engineering  Development  Report 

U5AF 

HT-S- 3582/ 
S-102-1 

».  description/purpose 

To  provide  visibility  to  the  Air  Force  program /project  man¬ 
ager  on  the  state  of  engineering  development  of  the  Cl/ 
subsystem,  as  well  as  provide  a  basis  for  projecting  needed 
supporting  efforts  such  as  verification  testing  and  production 
scheduling. 

«.  APPROVAL  DATE 

1  Nmrenfcer  1971 

S.  OFFICE  OF  PRIMARY 
responsibility 

AFSC 

G .  0 DC  RtOUIftCO 

t.  APPROVAL  limitation 

T.  APPLICATION/lNTERRELATIONSMIP 

These  data  are  related  to  the  design  requirements  (Part  1  of 
the  special)  for  the  n/subsystem  and  to  the  design  analyses 
conducted  on  the  item  during  the  design  process.  The  report 
may  consist  of  an  initial  report,  followed  by  updating  reports 
as  the  design  process  finalizes  the  design  configuration  of  the 
Cl/subsystem  by  the  time  of  the  physical  configuration 
audit  (PCA).  Although  acquired  primarily  during  sys- 
tems/major  equipment  asquisition  programs,  this  report  is 
also  applicable  to  other  efforts  where  information  on  design 
evolution  is  necessary  to  support  program  decisions  and  re¬ 
lated  developmental  efforts. 

t  REFERENCES  (Mmndmtory  Sm 
ciiad  in  block  10) 

MCSL  NUMBER'S! 


10.  PREPARATION  INSTRUCTIONS 

1.  The  contractor  shall  prepare  an  Engineering  Development  Report  for  each 
Cl/subs.vstem  as  specified  on  the  DD  Form  1423. 

2.  The  report  shall  be  structured  to  separately  cover  each  of  the  major  subtasks  of  the 
effort  required  to  develop  the  Cl/ subsystem,  as  defined  by  the  Design  Requirements 
Specification  and  contract. 

3.  The  report  will  describe  the  current  design  development  of  the  Cl/subsystem,  to 
include  such  information  as: 

a.  Predicted  performance  compared  to  and  based  upon  the  design  requirements 
characteristics.  Where  the  specific  performance  parameters  required  to  be  compared 
are  not  self-evident,  they  may  be  further  identified  in  an  attachment  to  the  CDRL  as  a 
modification  to  this  Data  Item  Description. 

b.  Status  of  the  configuration  design  and  changes  made  thereto  to  achieve  the 
required  performance  capability. 

c.  Discussion  of  design  and  development  investibations  (e.g.,  experiments  and  tests 
performed ) . 

d.  Design,  development,  and  /or  fabrication  problems  encountered,  with  proposed 

solutions  and  probable  impact. _ 

DD  ”-.1664 

Figure  13.  SUBSYSTEM/ENGINEERING  DEVELOPMENT  REPORT 
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2.4.5  Assessment  Technical  Performance  Measurement  Report 


(Dl-R-1754)  (U.S.  Army) 

The  TPM  report  "records  the  data  resulting  from  the  design 
assessment  which  estimates  and  measures  the  values  of  essential 
performance  parameters  of  the  current  design  of  the  product  Work 
Breakdown  Structure  (WBS)  elements.  The  data  are  used  as  a 
measure  of  the  effectiveness  of  actual  and  planned  system 
performance.  The  TPMR  is  initiated  during  contract  definition 
and  continues  throughout  the  development  and  production  phases. 
It  may  continue  into  the  operational  phase  if  product  improve¬ 
ment  takes  place"  (Figure  14). 

2.4.6  Assessment  System  Performance  Record  (DI-R-1755)  (U.S. 

Army) 

"The  System  Performance  Record  (SPR)  is  the  entire 
collection  of  data  which  must  be  maintained  in  order  to  explain 
the  methodology  and  criteria  for  system  performance  assessment, 
describe  the  system  WB.S,  identify  and  determine  the  status  of 
all  system  performance  parameters,  and  ensure  a  complete  analy¬ 
sis  of  the  effect  of  engineering  changes  on  system  performance" 
(Figure  15). 

2.4.7  Data  Accession  List/Internal  Data  ( UDI-A-26486 ) 


The  government  can  obtain  data  by  means  other  than  specific 
DID  inclusion  in  the  CDRL.  The  contractor  will  have  an  inter¬ 
nal  reporting  system  for  distributing  and  controlling  data  it 
produces.  The  government  can  order  data  from  the  list  provided 
to  the  PMO,  I  AW  UDI-A-26486,  of  these  internal  reports.  The 
reports  will  be  written  in  the  contractor's  format.  The  advan¬ 
tage  to  ordering  data  by  this  method  is  that  the  government  pays 
for  only  the  reproduction  costs  and  not  the  preparation  cost. 
The  disadvantage  of  this  method  is  the  lack  of  format  control 
and  extended  time  delay  in  delivery.  (Figure  16). 


DATA  ITEM  DESCRIPTION 


IDENTIFICATION  NO'S 


Assessment  Technical  Performance  Measurement  Report 


|»  OIICRlPTlOhPURPOlE 


Army  DI-R-1754 


4.  APPROVAL  DATE 

The  Technical  Performance  Measurement  (TPM)  Report  records  the  data  15  Dec  69 
resulting  from  the  design  assessment  which  estimates  and  measures  the  *  Sei»on£ »iLrTv*"’ 
values  of  essential  performance  parameters  of  the  current  design  of  the 
product  Work  Breakdown  Structure  (WBS)  elements.  The  data  is  used  USAMC 

as  a  measure  of  the  effectiveness  of  actual  and  planned' system  perform-  *  °°c  "tou,,,ED 


USAMC 


I?  APPwICATION/INTIRRCL  AT  ION  4m  I  P 


The  TPM  is  initiated  during  contract  definition  and  continues  through¬ 
out  the  development  and  production  phases.  It  may  continue  into  the 
operational  phase  if  product  improvement  takes  place. 

DI-R-1750,  Assessment  Program  Plan 

DI-R-1753,  Assessment  System  Performance  Status  Report 


!•  APPROVAL  LIMITATION 


».  REPCRENCCS  (/HMdAiery  •<  ci(ed  in 
fcJoc*  J  0) 

AR  70-32 
MIL-STD-881 
TM  38-760 


MdL  NUMIENOI  509 

10100  30854 


tC  PREPARATION  INSTRUCTIONS 

1  .  Unless  specified  otherwise  by  the  procuring  activity  the  TPM  Report  shall  contain  the  following 
sections: 

a.  Planned  Parameter  Profiles 

b.  Parameter  Status  Tracking  and  Forecast 

c.  Recoras  of  Achieved  Parameter  Profiles 

d.  Probl  em  Analysis  and  Corrective  Action 

e.  Input  Data  for  the  System  Performance  Status  Report  (SPSR). 

2.  Instructions  for  the  preparation  of  each  of  these  sections  will  be  as  specified  by  the  procuring 
activity . 


DD  .'25*..  1664 


Figure  14.  ASSESSMENT  TPM  REPORT 
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DATA  ITBM  DCSCttmON 


Assessment  System  Performance  Record 


IDCHTiriCATlOM  MOISI 


DI-R-1755 


•  OCDC*«*riOM/»Uft»OftK 


The  System  Performance  Record  (SPR)  is  the  entire  collection  of  data 
which  must  be  maintained  in  order  to  explain  the  methodology  and 
criteria  for  system  performance  assessment ,  describe  the  system  Waste 
Breakdown  Structure  (WBS),  identify  and  determine  the  status  of  all 
system  performance  parameters  ,  and  ensure  a  complete  analysis  of  the 
effect  of  engineering  changes  on  system  performance. 


15  Dec  69 


USAMC 
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The  SPR  represents  the  data  base  from  which  System  Performance  Status 
Reports  (Di-R-1753)  are  prepared. 

DI-R-1750,  Assessment  Program  Plan 


Mm*  |0J 

MIL-STD-881 
AR  70-32 
TM  38-760 
AMCR  702-8 


«0  PPIPARATION  IlMTIlUCTIONa 


The  SPR  shall  be  structured  and  maintained  as  specified  by  the  procuring  activity. 
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Figure  15.  ASSESSMENT  SYSTEM  PERFORMANCE  RECORD 


09  1 - » AA»t 


DATA  ITEM  DESCRIPTION 


IDENTIFICATION  NOISI 


DATA  ACCESS  I  Oh  LIST /INTERNAL  DATA 


»  DIICAKTION/FUMOK 


The  purpose  of  the  data  item  description  is  to 
provide  an  accession  list  which  is  an  index  of  dati 
that  may  be  available  for  request.  It  is  a  medium 
for  identifying  contractor  internal  data  which 
have  been  generated  by  the  contractor  in  complianci 
with  the  work  effort  described  in  the  Statement  of 
Work. 


ANNOC  AT  ION/ IN  TERKCt  AT  IONIHIP 


AVY-SH  I UDI-A-2 64 86 


APPROV AL  DATE 

73  Dec  14 


7.1  This  Data  Item  Description  is  designed  for  use 
on  contracts  to  facilitate  the  Identification  of 
internally  generated  data  that  is  usually  not 
determinable  at  the  outset  of  a  contract.  The  lisi 
may  be  used  to  identify  data  developed  by  the 
contractor  for  his  own  internal  use  that  may  be 
of  value  in  Government  program  management. 


•  REFCRENCO  fUfcmctalory  ••  clltd  in 
block  10) 


MCIL  NUMBENID 


10  PREP  AASTIQN  INSTRUCTIONS 


10.1  The  Data  Accession  List  is  a  list  of  contractor  internally  gener¬ 
ated  data  used  by  the  contractor  to  develop,  test,  and  manage  a  program. 
The  format  and  content  of  these  data  shall  be  as  prepared  by  the  con¬ 
tractor  to  document  his  compliance  with  the  Statement  of  Work  task 
requirements . 

10.2  The  Data  Accession  List  shall  be  a  listing  of  all  data,  including 
drawings,  documents,  specifications,  manuals,  plans,  reviews,  reports, 
computer  program,  and  vendor- furnished  data  developed  under  a  program 
contract,  whether  technical  or  managerial,  deliverable  or  non-deliver¬ 
able. 

10.2.1  The  list  shall  include  as  a  minimum:  the  identification  number, 
title,  and  in-house  release  date,  f orecast/actual . 

10.2.2  Data  items  shall  be  cross  reference  to  their  appropriate  Con¬ 
tract  Work  Breakdown  Structure  sub-task  number.  The  contractor  or¬ 
ganizational  unit  responsible  for  each  data  item  will  be  included. 

10.2.3  The  list  shall  begin  as  a  listing  of  the  contractual  (as 
negotiated)  Authorized  Data  List  and  shall  develop  to  include  all  data 
generated  in  the  program. 
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Figure  16.  DATA  ACCESSION  LIST/INTERNAL  DATA 


The  Data  Requirements  Review  board  will  develop  the  list  of 
data/reports  that  the  program  will  put  on  contract  through  the 
CDRL.  The  PM  should  make  the  final  decision  on  data  require¬ 
ments  based  on  cost  and  need.  For  TPM,  parameters  will  be 
selected,  identified  in  the  SOW,  and  reports  will  be  required. 
As  noted  in  paragraph  2.3.4,  parameters  will  be  selected  for 
various  reasons;  however,  requirements  levied  by  higher  head¬ 
quarters  must  be  considered  when  data  requirements  are  devel¬ 
oped.  The  previous  listed  DIDs  provide  the  basis  for  TPM 
reporting;  however,  specially  tailored  reports  can  be  devel¬ 
oped.  Some  contractors  have  established  TPM  reporting  for¬ 
mats.  If  these  are  appropriate,  they  may  be  requested  through 
the  Data  Accession  Listing. 

As  parameters  are  achieved  and  no  further  action  planned, 
reporting  requirements  should  be  deleted.  Contractor  reporting 
frequency  should  be  based  on  the  expected  change  in  data  and 
the  PMO's  need  for  information.  Contractor  reports  have  a  cost 
and  should  only  be  required  or  continued  when  they  provide 
needed  information  for  timely  PMO  action. 
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COST/SCHEDULE  CONTROL  SYSTEM  CRITERIA  (C/SCSC) 

3.1  INTRODUCTION 

A  fundamental  responsibility  in  the  acquisition  and  modifi¬ 
cation  of  major  systems  is  to  ensure  that  visibility  of  con¬ 
tractors'  progress  is  sufficient  to  reliably  assess  the  results 
being  obtained.  C/SCSC  allows  contractors  to  use  the  specific 
management  system  of  their  choice.  The  use  of  generalized 
criteria  establishes  the  characteristics  and  capabilities  which 
should  be  inherent  in  an  effective  cost  and  schedule  control 
system  as  an  alternative  to  a  single  DoD  dictated  measurement 
system.  The  objective  is  to  obtain  assurance  that  the  contrac¬ 
tor's  internal  management  systems  are  sound,  and  to  obtain 
summarized  data  for  cost  effective  contract  management.  This 
chapter  is  designed  to  provide  basic  knowledge  of  C/SCSC  and 
the  reports  that  are  generated  from  an  acceptable  contractor 
system.  For  further  information,  the  following  documents 
should  be  reviewed:  DoDI  7000.2  Performance  Measurement  for 
Selected  Acquisitions;  AFSCP  173-5,  DARCOM-P  715-5,  NAVMAT 
P5240,  Cost/Schedule  Control  Systems  Criteria  Joint  Implementa¬ 
tion  Guide;  and  DoDI  7000.10,  Contract  Cost  Performance,  Funds 
Status  and  Cost/Schedule  Status  Reports. 

3.2  THE  CRITERIA 


DoD  Instruction  7000.2,  Enclosure  1,  Performance  Measure¬ 
ment  for  Selected  Acquisition,  and  the  Tri-Service  Cost/Sched¬ 
ule  Control  Systems  Criteria,  Joint  Implementation  Guide, 
provide  the  guidance  for  implementation  of  C/SCSC. 


After  considerable  turmoil,  DoD  determined  that  the  devel¬ 
opment  of  a  universal  system  for  cost  and  schedule  control  was 
not  feasible.  No  single  set  of  management  control  systems  will 
meet  every  DoD  and  contractor  need  for  performance  measurement 
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due  to  variations  in  organizations,  products  and  methods  util¬ 
ized.  DoD  has  adopted  an  approach  which  simply  defines  the 
criteria  that  must  be  met  by  contractors'  management  control 
systems. 

As  a  minimum,  contractors'  management  control  systems  are 
expected  to  provide  a  framework  for  defining  work;  assigning 
work  responsibility;  establishing  budgets;  controlling  costs; 
and  summarizing,  with  respect  to  planned  versus  actual  accom¬ 
plishment,  the  detailed  cost,  schedule,  and  related  technical 
achievement  information  for  appropriate  management  levels. 
Such  systems  must  provide  for: 

o  Realistic  budgets  for  work  schedule  within  responsibil¬ 
ity  assignments. 

o  Accurate  accumulation  of  costs  related  to  progress  of 
the  planned  work. 

o  Comparison  between  the  actual  resources  applied  and  the 
estimated  resources  planned  for  specific  work  assign¬ 
ments. 

o  Preparation  of  reliable  estimates  of  costs  to  complete 
remaining  work. 

o  Support  an  overall  capability  for  managers  to  analyze 
available  information  to  identify  problem  areas  in 
sufficient  time  to  take  remedial  action. 

The  criteria  are  grouped  under  five  headings:  ORGANIZATION; 
PLANNING  &  BUDGET;  ACCOUNTING;  ANALYSIS;  and  REVISIONS  &  ACCESS 
TO  DATA.  The  contractor's  management  control  systems  will 
include  policies,  procedures  and  methods  designed  to  ensure 
that  they  will  accomplish  each  of  the  five  items. 


3.2.1.  Organization 

o  'Define  all  authorized  work  and  related  resources  to 
meet  the  requirements  of  the  contract,  using  the  frame¬ 
work  of  the  Contractor  Work  Breakdown  Structure  (CWBS). 

o  Identify  the  internal  organizational  elements  and  the 
major  subcontractors  responsible  for  accomplishing  the 
authorized  work. 
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o  Provide  for  the  integration  of  the  contractor's  plan¬ 
ning,  scheduling,  budgeting,  work  authorization  and  cost 
accumulation  systems  with  each  other,  the  CWBS,  and  the 
organizational  structure. 

o  Identify  the  managerial  positions  responsible  for  con¬ 
trolling  overhead  (indirect  costs)." 

3.2.2  Planning  and  Budgeting 

o  "Schedule  the  authorize  work  in  a  manner  which  describes 
the  sequence  of  work  and  identifies  the  significant  task 
interdependencies  required  to  meet  the  development, 
production,  and  delivery  requirements  of  the  contract. 

o  Identify  physical  products,  milestones,  technical  per¬ 
formance  goals,  or  other  indicators  that  will  be  used  to 
measure  output. 

o  Establish  and  maintain  a  time-phased  budget  baseline  at 
the  cost  account  level  against  which  contract  perform¬ 
ance  can  be  measured.  Initial  budgets  established  for 
this  purpose  will  be  based  on  the  negotiated  target 
cost.  Any  other  amount  used  for  performance  measurement 
purposes  must  be  formally  recognized  by  both  the  con¬ 
tractor  and  the  Government. 

o  Establish  budgets  for  all  authorized  work  with  separate 
identification  of  cost  elements  (labor,  material,  etc.). 

o  To  the  extent  that  the  authorized  work  can  be  identified 
in  discrete,  short-span  work  packages,  establish  budgets 
for  this  work  in  terms  of  dollars,  hours,  or  other 
measurable  units.  Where  the  entire  cost  account  cannot 
be  subdivided  into  detailed  work  packages,  identify  the 
far-term  effort  in  larger  planning  packages  for  budget 
and  scheduling  purposes. 

o  Provide  that  the  sum  of  all  work  package  budgets  plus 
planning  packages  within  a  cost  account  equals  the  cost 
account  budget. 

o  Identify  relationships  of  budgets  or  standards  in  under¬ 
lying  work  authorization  systems  to  budgets  for  work 
packages. 

o  Identify  and  control  Level  of  Effort  (LOE)  activity  by 
time-phased  budgets  established  for  this  purpose.  Only 
that  effort  which  cannot  be  identified  as  discrete, 
short-span  work  packages  or  as  apportioned  effort  will 
be  classified  as  level  of  effort. 

(4)  DARCOM-P  715-13/NAVMAT  P5244/AFLCP  173-2/AFSCP  173-3/ 

DLAH  8315.3,  Cost/Schedule  Control  System  Criteria, 
Joint  Implementation  Guide,  October  1980. 
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o  Establish  overhead  bud~2ts  for  the  total  costs  of  each 
significant  organizational  component  whose  expenses  will 
become  indirect  costs.  Reflect  in  the  contract  budgets, 
at  the  appropriate  level,  the  amounts  in  overhead  pools 
that  will  be  allocated  to  the  contract  as  indirect  costs. 

o  Identify  management  reserves  and  undistributed  budget. 

o  Provide  that  the  contract  target  cost  plus  the  estimated 
cost  of  authorized  but  unpriced  work  is  reconciled  with 
the  sum  of  all  internal  contract  budgets  and  management 
reserves.* 

3.2.3  Accounting 

o  "Record  direct  costs  on  an  applied  or  other  acceptable 
basis  in  a  formal  system  that  is  controlled  by  the 
general  books  of  accounting. 

o  Summarize  direct  costs  from  cost  accounts  into  the  WBS 
without  allocation  of  a  single  cost  account  to  two  or 
more  WBS  elements. 

o  Summarize  direct  costs  from  the  cost  accounts  ihto  the 
contractor's  functional  organizational  elements  without 
allocation  of  a  single  cost  account  to  two  or  more 
organizational  elements. 

o  Record  all  indirect  costs  which  will  be  allocated  to  the 
contract. 

o  Identify  the  basis  for  allocating  the  cost  of  appor¬ 
tioned  effort. 

o  Identify  unit  costs,  equivalent  unit  costs,  or  lot 
costs,  as  applicable. 

o  The  contractor's  material  accounting  system  will  provide 
for : 

Accurate  cost  accumulation  and  assignment  of  costs  to 
cost  accounts  in  a  manner  consistent  with  the  budg¬ 
ets,  using  recognized,  acceptable  costing  techniques. 

Determination  of  price  variances  by  comparing  planned 
versus  actual  commitments. 

Cost  performance  measurement  at  the  point  in  time 
most  suitable  for  the  category  of  material  involved, 
but  no  earlier  than  the  time  of  actual  receipt  of 
material . 

Determination  of  cost  variances  attributable  to  the 
excess  usage  of  material. 
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Determination  of  unit  or  lot  costs  when  applicable 


3.2.4  Analysis 

o  "Identify  at  the  cost  account  level  on  a  monthly  basis 
using  data  from,  or  reconcilable  with,  the  accounting 
system: 

Budgeted  cost  for  work  scheduled  and  budgeted  cost 
for  work  performed. 

Budgeted  cost  for  work  performed  and  applied  (actual 
where  appropriate)  direct  costs  for  the  same  work. 

Variances  resulting  from  the  above  comparisons  clas¬ 
sified  in  terms  of  labor,  material,  or  other  appro¬ 
priate  elements  together  with  the  reasons  for  signif¬ 
icant  variances. 

o  Identify  on  a  monthly  basis,  in  the  detail  needed  by 

management  for  effective  control,  budgeted  indirect 
costs,  actual  indirect  costs,  and  variance  along  with 
the  reasons. 

o  Summarize  the  data  elements  and  associated  variances 

listed  above,  through  the  contractor  organization  and 
WBS  to  the  reporting  level  specified  in  the  contract. 

o  Identify  significant  differences  on  a  monthly  basis 

between  planned  and  actual  schedule  accomplishment  and 
the  reasons  for  the  discrepancies. 

o  Based  on  performance  to  date  and  on  estimates  of  future 
conditions,  develop  revised  estimates  of  cost  at  comple¬ 
tion  for  WBS  elements  identified  in  the  contract,  and 

compare  these  with  the  contract  budget  base  and  the 
latest  statement  of  funds  requirements  reported  to  the 
Government." 

3.2.5.  Revisions  and  Access  to  Data 


o  "Incorporate  contractual  changes  in  a  timely  manner, 
recording  the  effects  of  such  changes  in  budgets  and 
schedules.  In  the  directed  effort  before  negotiation  of 
a  change,  base  such  revisions  on  the  amount  estimated 
and  budgeted  to  the  functional  organizations. 


o  Reconcile  original  budgets  for  those  elements  of  the  WBS 
identified  as  priced  line  items  in  the  contract,  and  for 
those  elements  at  the  lowest  level  of  the  DoD  Project 
Summary  WBS,  with  current  performance  measurement  budg¬ 
ets  in  terms  of  (a)  changes  to  the  authorized  work  and 
(b)  internal  replanning  in  the  detail  needed  by  manage¬ 
ment  for  effective  control. 

o  Prohibit  retroactive  changes  to  records  pertaining  to 
work  performed  that  will  change  previously  reported 
amounts  for  direct  costs,  indirect  costs,  or  budgets, 
except  for  correction  of  errors  and  routine  accounting 
adjustments . 

o  Prevent  revisions  to  the  contract  budget  base  except  for 
Government-directed  changes  to  contractual  effort. 

o  Document,  internally,  changes  to  the  performance  meas¬ 
urement  baseline  and,  on  a  timely  basis,  notify  the 
procuring  activity  through  prescribed  procedures. 

o  Provide  the  contracting  officer  and  duly  authorized 
representatives  access  to  all  of  the  foregoing  informa¬ 
tion  and  supporting  documents.*  (8) 


3.3  HOW  IS  COST  AND  SCHEDULE  PERFORMANCE  MEASURED? 


The  objective,  of  C/SCSC  is  twofold:  first  to  obtain  assur¬ 
ance  that  contractors'  internal  management  control  systems  are 
sound,  and  second  to  obtain  summarized  data  for  contract  man¬ 
agement.  Contractors'  internal  systems  must  be  able  to  provide: 

o  Budgeted  Cost  for  Work  Scheduled  (BCWS). 

o  Budgeted  Cost  for  Work  Performed  (BCWP). 

o  Actual  Cost  of  Work  Performed  (ACWP). 
o  Estimated  cost  at  completion, 
o  Budgeted  cost  at  completion, 

o  Cost  and  schedule  variances, 

o  Traceability. 


I  8) ibid 


The  BCWS  represents  the  value  of  the  work  planned  to  be  done 
at  a  given  point  in  time.  BCWP  represents  the  value  of  com¬ 
pleted  work.  The  comparison  of  BCWS  with . BCWP  indicates  whether 
more  or  less  work  was  done  than  scheduled:  the  difference  is 
schedule  variance.  Comparison  of  BCWP  with  ACWP  indicates 
whether  the  actual  cost  of  the  work  performed  was  greater  or 
less  than  the  planned  cost.  Based  on  performance  to  date  and 
estimate  of  future  conditions,  an  estimated  cost  at  completion 
can  be  computed  and  compared  to  the  total  budget.  At  the  con¬ 
tract  level,  total  budget  is  usually  equal  to  the  contract 
value,  and  the  difference  will  provide  a  forecast  of  contract 
over  or  under-run. 

The  WBS,  MIL-STD-881A,  is  a  family-tree  type  subdivision  of 
products,  components,  work  tasks  and  services  required  to  pro¬ 
duce  an  end  product  (Figure  17).  For  performance  measurement 
purposes,  it  is  desirable  that  the  WBS  be  structured  in  the  same 
way  that  the  work  is  actually  to  be  performed.  The  top  three 
levels  of  the  contract  WBS  are  developed  by  the  government  and 
negotiated  with  the  contractor.  These  summary  level  items  are 
included  in  the  contract.  The  contractor  may  extend  the  summary 
WBS  in  any  manner  he  chooses  to  divide  the  contractual  work  into 
manageable  portions,  before  he  assigns  responsibility.  Contract 
line  items  are  included  normally  as  separate  WBS  elements  and 
the  WBS  is  aligned  with  the  statement  of  work  as  much  as  possi¬ 
ble.  Integration  of  the  contractor  organizational  structure 
with  the  WBS  is  necessary  in  order  to  assign  functional  respon¬ 
sibility  (Figure  18).  The  intersection  of  the  organizational 
structure  and  the  CWBS  is  normally  referred  to  as  the  cost 
account ,  and  it  is  at  this  point  that  collection  and  analysis  of 
cost  and  other  information  is  accomplished  prior  to  summation 
for  higher  level  management. 

The  integration  of  the  organizational  structure  and  the  CWBS 
results  in  a  key  intersection  or  management  control  point  (cost 
account).  Integration  of  the  other  subsystems  (scheduling, 
budgeting,  work  authorization,  and  cost  collection)  should  also 
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Figure  17.  WORK  BREAKDOWN  STRUCTURE 


Figure  18.  INTEGRATION 


exist  at  the  management  control  point  since  this  is  where  the 
planning  and  management  for  lower  level  tasks  occurs. 

Once  functional  responsibilities  for  the  work  have  been 
established/  further  subdivision  of  the  effort  into  work  pack¬ 
ages  can  be  accomplished  and  the  work  can  be  identified  with  the 
performing  organization  or  individuals. 

Work  packages  are  the  basic  building  blocks  used  for  de¬ 
tailed  planning,  assignment  and  control  of  contract  perform¬ 
ance.  The  task  of  a  work  package  should  be  clearly  defined, 
scheduled,  budgeted  and  assigned  to  a  single  organization  re¬ 
sponsible  for  its  completion. 

The  contractor  defines  the  contractual  effort  using  the  WBS 
.as  an  aid  to  subdividing  and  displaying  units  of  work.  Schedul¬ 
ing  and  budgeting  the  work  produces  a  time-phased  baseline  which 
can  be  used  for  performance  measurement  purposes,  and  effec¬ 
tively  integrates  the  work,  schedule  and  budget.  Contract 
accomplishment  is  measured  by  accumulating  the  budgets  applica¬ 
ble  to  work  performed  (BCWP)  and  comparing  them  to  the  budgets 
for  work  scheduled  (BCWS)  and  to  the  actual  costs  of  work  per¬ 
formed  ( ACWP ) .  This  derives  both  schedule  and  cost  variances. 
Performance  trends  and  work  yet  to  be  accomplished  can  be  used 
to  develop  an  estimated  cost  at  completion. 

3.4  REPORTING 

As  noted  earlier,  C/SCSC  provides  criteria  that  contractors' 
management  control  systems  must  meet.  C/SCSC  is  not  a  reporting 
system.  Department  of  Defense  Instruction  7000.10,  Contract 
Cost  Performance,  Cost/Schedule  Status  Reports,  provides  proce¬ 
dures  for  collecting  summary  level  cost  and  schedule  performance 
data  from  contractors  for  program  management  purposes.  This 
instruction  encourages  contractors  to  substitute  internal  re¬ 
ports,  provided  that  the  data  elements  and  definitions  are 
compatible  with  prescribed  requirements  and  in  a  form  suitable 
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for  management  use.  Data  Item  Description  DI-F-6000B  identifies 
the  Cost  Performance  Report  (Figure  19). 

3.4.1  Cost  Performance  Report  (CPR) 

The  CPR  provides  contract  cost/schedule  status  information 
for  use  in  making  and  validating  management  decisions  based  on 
problem  identification  through  variance  analysis  of  both  cost 
and  schedule.  The  CPR  is  provided  in  five  formats: 

Format  1  provides  data  to  measure  cost  and  schedule  perform¬ 
ance  by  summary  level  WBS  elements  (Figure  20). 

Format  2  provides  similar  measurement  by  organizational  or 
functional  categories  (Figure  21). 

Format  3  provides  the  budget  baseline  plan  against  which 
performance  is  measured  (Figure  22). 

Format  4  provides  manpower  loading  forecasts  correlated  with 
the  budget  plan  and  cost  estimate  predictions  (Figure  23). 

Format  5  is  a  narrative  report  used  to  explain  significant 
cost  and  schedule  variances  and  other  identified  contract  prob¬ 
lems  (Figure  24). 

Program  managers  use  the  CPR  to  evaluate  contract  perform¬ 
ance,  identify  the  size  and  impact  of  actual  and  potential 
problem  areas  causing  significant  cost  and  schedule  variance, 
and  provide  status  information  to  higher  headquarters.  . 

3.4.2  Cost/Schedule  Status  Report  (C/SSR) 

The  C/SSR  provides  summarized  cost  and  schedule  performance 
information  which  is  limited  to  level  3  or  higher  of  the  con¬ 
tract  work  breakdown  structure.  It  is  normally  used  on  smaller 
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ft.  DUCRIRTION/PURROII 

3.1  This  report  is  prepared  by  contractors  and  constats  of 
five  formats  containing  cost  and  related  data  for  measuring 
contractors'  cost  and  schedule  performance.  Format  1  pro¬ 
vides  data  to  measure  cost  and  schedule  performance  by 
summary  level  work  breakdown  atructure  elements.  Format  2 
provides  a  similar  measurement  by  organizational  or  functional 
cost  categories.  Format  3  provides  the  budget  baseline  plan 
against  which  performance  is  measured.  Format  A  provides 
manpower  loading  forecasts  for _ (Continued  on  page  2) 


T.  APNLIC  ATION/INTKMNKL  ATIONAMIA 

7.1  The  CPR  will  normally  be  required  for  selected  contracts 
within  those  programs  designated  as  major  programs  in 
accordance  with  DoD  Directive  5000. 1,  "Major  System  Acquisi¬ 
tions,"  dated  18  January  1977.  It  will  be  established  as 

a  contractual  requirement  as  aet  forth  in  the  DD  Form  1423 
Contract  Data  Requirements  List  (CDRL),  and  DD  Form  1660, 
Management  System  Sutmaary  List. 

7.2  If  the  CPR  supports  a  contractual  requirement  for  con¬ 
tractor  compliance  with  the  Cost/Schedule  Control  Systems 
Criteria  (C/SCSC),  the  CPR  data  elements  will  reflect  the 
contractor's  implementation  in  accordance  with  Department  of 
Defense  Instruction  7000.2.  If  compliance  with  the  C/SCSC 

(Continued  on  page  2) 


M.  AH  AT  ION  IMTHUCTieai 

10.1  Unless  otherwise  stated  in  the  solicitation,  the  effective  issue  of  the  docu¬ 
ment  (s)  cited  in  the  referenced  document (s)  in  this  block  shall  be  that  listed  in 
the  issue  of  the  DoD  Index  of  Specifications  and  Standards  (DoDISS)  and  the  supple¬ 
ments  thereto  specified  in  the  solicitation  and  will  form  a  part  of  this  data  item 
description  to  the  extent  defined  within. 

10.2  Hard  copy  printouts  from  contractors'  Internal  mechanized  reporting  systems 
may  be  substituted  for  CPR  formats  provided  the  printouts  contain  all  the  required 
data  elements  at  the  specified  reporting  levels  in  a  form  suitable  for  DoD  manage¬ 
ment  use.  Where  data  are  furnished  which  require  mechanized  processing,  narrative 
remarks  should  accompany  capes  or  cards  and  Identify  pertinent  items  to  which  they 
apply,  and  a  listing  of  the  tape  or  card  data  should  be  included  to  expedite  pro¬ 
cessing.  CPR  formats  will  be  completed  in  accordance  with  the  following  instructions: 

10.2.1  Heading  Information  -  Formats  1  through  4 


DoDD  5000.1 
DoDD  5000.19 
DoDD  5000.32 
DoDI  7000.2 
DoDI  7000.10 
Cost  Accounting 
Standard  414 


■CM.  NIMMAIA 


10.2.1.1  Contractor  Name  and  Location:  Enter  the  name,  division,  if  applicable, 
plant  location  and  mailing  address  of  the  reporting  contractor. 

10.2.1.2  RDT&E  r __j  Production  /.  /:  Check  appropriate  box.  Separate  reports 

are  required  for  each  type  of  contract. 


10.2.1.3  Contract  Type/Number:  Enter  the  contract  type,  contract  number  and 
the  number  of  the  latest  contract  change  or  supplemental  agreement  applicable  to 
the  contract. 

(Continued  on  page  3) 


Figure  19.  COST  PERFORMANCE  REPORT 


COST  PERFORMANCE  REPORT  -  WORK  BREAKDOWN  STRUCTURE 


Figure  21.  CPR  -  FUNCTIONAL  CATEGORIES 


COST  PERFORMANCE  REPORT  -  BASELINE 


DI-F-6000B 


Figure  22.  CPR  -  BASELINE 
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Figure  24.  CPR  -  PROBLEM  ANALYSIS 


programs  which  do  not  use  the  CPR  and  is  provided  in  only  a 
single  format,  versus  the  five  formats  for  the  CPR.  The  C/SSR 
does  not  have  the  underlying  requirement  -  of  a  compliant  C/SCSC 
system  as  does  the  CPR,  therefore  knowledge  of  the  contractor 
system  is  extremely  important  to  the  PM.  Data  Item  Description 
DI-F-601QA  identifies  the  C/SSR  (Figure  25).  Figure  26  provides 
a  sample  of  the  C/SSR  Report.  For  further  information  on  the 
C/SSR  refer  to  the  Cost/Schedule  Management  of  Non-Major  Con¬ 
tracts  (C/SSR  Joint  Guide)  1  Nov.  78. 


DATA  ITEM  DESCRIPTION 
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COST/SCHEDULE  STATUS  REPORT  (C/SSR) 


3.1  This  report  is  prepared  by  contractors  and  provides’ 
summarized  cost  and  schedule  performance  information  for 
program  management  purposes. 


AGENCY 

NutfBCA 

OSD 

D1-F-6C1 OA 

4  APPROVAL  DAT  E 

1  November  1979 

•  ODC  required 


IP^KO  V1L  limitation 


AAALICAT«N/INT(«AtLATION>HIP 

7.1  The  Cost/Schedule  Status  Report  (C/SSR),  Figure  1,  is 
applicable  to  contracts  of  $2,000,000  or  over  and  12  months' 
duration  or  more  which  do  not  use  the  Cost  Performance  Report 
(DI-F-6000) .  It  will  be  established  as  a  contractual 
requirement  as  set  forth  in  the  Contract  Data  Requirements 
List,  DD  Form  1423,  and  Management  System  Summary  List, 

DD  Form  1660. 
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DoD  4120.3M,  Aug  76 
MIL  STD  681A,  25  Apr  75 
DoDI  7000.2,  10  Jur,  77 


7.2  Data  reported  on  the  C/SSR  will  pertain  to  all  author¬ 
ized  contract  work,  including  both  priced  and  unpriced  effort. 
Data  reported  will  be  limited  to  level  3  of  the  contract  work 
breakdown  structure  or  higher.  However,  if  a  problem  area  is 
indicated  at  a  lower  level,  more  detailed  data  will  be  pro¬ 
vided  on  an  exception  basis  until  the  problem  is  resolved. 
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M  PREPARATION  INSTRUCTIONS 

10.1  Unless  otherwise  stated  in  the  solicitation,  the  effective  issue  of  the  docu¬ 
ment^)  cited  in  the  referenced  document(s)  in  this  block  shall  be  that  listed  in 
the  issue  of  the  DoD  Index  of  Specifications  and  Standards  (reference  DoD  4120. 3M)  anc 
the  supplements  thereto  specified  in  the  solicitation  and  will  form,  a  part  of  this 
data  item  description  to  the  extent  defined  within. 


10.2  Heading  Information 


10.2.1  CONTRACTOR :  Enter  the  lame  and  division  (if  applicable)  of  the  reporting 
contractor. 


10.2.2  LOCATION :  Enter  the  plant  location  and  mailing  address. 


10.2.3  RDT&E  ^*~7  PRODUCTION  /~~7 ■  Check  appropriate  box.  Separate  reports 
are  required  for  each  type  of  contract. 


10.2.4  CONTRACT  TYPE  AND  NUMBER :  Enter  the  contract  type,  contract  number  and 
the  number  of  the  latest  contract  change  order  or  supplemental  agreement  applicable 
to  the  contract. 


10.2.5  PROGRAM  NAME/NUMBER:  Enter  the  name,  number,  acronym  and/or  the  type, 
model  and  series,  or  other  designation  of  the  prime  items  purchased  under  the  contract. 
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Figure  25.  COST/SCHEDULE  STATUS  REPORT 
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Chapter  4 


TPM  CURRENT  ACTIVITIES 


4.1  INTRODUCTION 


An  interview  survey  was  completed  with  the  engineering 
community  within  the  DoD  PM  community  and  with  Defense  Contrac¬ 
tors  (Appendix  C).  All  organizations  interviewed  were  directly 
involved  with  the  acquisition  or  development  of  products  for 
DoD.  The  objective  was  to  determine  the  means  by  which  the 
goals  of  TPM  are  currently  being  accomplished. 

Government  engineers  were  interviewed  at  all  organizational 
levels  within  the  acquisition  community.  Position  titles  var¬ 
ied,  among  them  were;  Program  Manager,  Chief  Engineer,  Director 
for  Engineering,  Deputy  for  Engineering,  Associate  Project 
Leader  and  functional  engineer.  Program  size  and  complexity 
were  varied  as  well,  with  the  number  of  engineers  assigned 
ranging  from  several  hundred  to  one  or  two.  Some  programs  had 
engineers  residing  within  the  Program  Office  while  other  pro¬ 
grams  had  engineers  remaining  within  their  functional  area  and 
providing  support  to  the  program  office.  Engineering  support 
from  functional  specialty  areas  ranged  from  dedication  to  a 
single  program  to  multiple  program  support. 

The  Defense  Contractor  engineering  interviews  were  performed 
with  contractors  who  had  at  least  50%  of  their  business  with  the 
Department  of  Defense.  Again,  the  interviews  were  designed  to 
check  the  view  from  all  engineering  levels.  In  most  cases 
several  individuals  were  interviewed  simultaneously,  each  pro¬ 
viding  their  perspective  on  TPM  and  the  CPR.  Position  titles 
spanned  the  range  from  Vice  President  for  Engineering  and  Tech¬ 
nical  Support,  Program  Manager  for  Engineering,  Laboratory 
Manager,  Associate  Engineering  Director,  Deputy  Director  for 
Engineering,  to  Avionics  Design  Manager  and  Weight  Manager. 
Organizational  positions  ranged  from  program  office  to  func¬ 
tional  area  specialists. 


Questions  were  developed  for  each  of  the  two  groups  -  gov¬ 
ernment  and  contractor  engineers.  The  interviews  with  the 
government  engineers  concentrated  on  their  methods  of  monitoring 
their  contractors'  technical  performance  while  the  interviews 
with  the  contractors  concentrated  on  their  methods  of  measuring 
their  technical  progress  and  providing  government  required 
information.  Although  responses  varied,  patterns  developed 
which  established  the  prevalent  practices  and  perceptions  within 
the  two  communities. 

This  chapter  details  the  survey  results,  provides  some 
observations,  and  forms  the  basis  for  Chapter  5  which  offers  the 
PM  guidelines  for  an  effective/affordable  TPM  program  which 
reflects  current  practice  and  experience. 


4.2  SERVICE  PROGRAM  OFFICE  POINT  OF  VIEW 


4.2.1  Monitoring  Technical  Performance 


The  key  question  asked  of  the  government  acquisition  engi¬ 
neers  was  how  they  monitor  their  contractors'  technical  perform¬ 
ance.  Once  a  program  has  hardware  to  test  the  answer  naturally 
becomes  the  monitoring  of  test  results.  The  more  difficult 
challenge  was  the  monitoring  during  D/V  and  the  early  stages  of 
FSD  when  the  design  is  still  being  iterated  and  trade-offs  are 
being  accomplished.  The  interviewees  consistently  stated  that 
technical  progress  is  monitored  through  daily  interface  or 
technical  interchange  meetings  held  at  the  contractors  plant, 
where  a  one  on  one  engineering  exchange  takes  place.  This 
occurs  at  all  organizational  levels  and  is  generally  held  at  the 
contractor's  plant  due  to  the  reduced  cost  involved  in  bringing 
government  engineers  to  the  plant  rather  than  bringing  the 
numerous  contractor  engineers  to  the  PMO;  and  because  the  gov¬ 
ernment  engineers  need  to  see  the  results  of  the  design  and  test 
activity  as  it  progresses.  In  some  cases,  based  on  the  PM's 
management  style,  government  engineers  will  be  assigned  to  the 


contractor's  plant  during  critical  periods  to  maintain  vigilance 
over  the  activity.  The  PM  may  feel  that  he  needs  his 
representative(s)  on  site  full  time  to  .  ensure  that  he  stays 
adequately  informed  of  the  contractor's  progress  due  to  the 
inherent  time  delay  in  any  formal  reporting  system. 

Based  on  the  significance  that  the  government  engineers 
placed  on  face  to  face  meetings  with  the  contractor's  engineers 
to  evaluate  technical  progress,  the  next  question  involved 
utilization  of  the  existing  Plant  Representative  Office  (PRO). 
The  Naval  Plant  Representative  Office  (NAVPRO),  Air  Force  Plant 
Representative  Office  (APPRO),  and  the  Defense  Contract  Admin¬ 
istrative  Service  (DCAS),  are  the  PM's  agent  in  the  plant  and 
appear  to  be  a  logical  means  of  monitoring  technical  perform¬ 
ance.  All  PMO's  interviewed  had  PRO  organizations  supporting 
their  programs  and  their  role  was  defined  within  regulations. 
The  PRO  monitored  the  contractor's  compliance  with  the  contract 
and  the  quality  of  the  product  being  produced.  They  were  not 
involved  in  the  monitoring  of  technical  progress  in  the  develop¬ 
ment  (design)  of  the  product.  The  PRO  organization  had  limited 
resources  and  TPM  was  not  in  their  charter.  In  some  cases. 
Memorandums  Of  Agreement  (MOAs)  had  been  developed  between  the 
PMO  and  the  PRO  to  perform  other  functions,  but  in  no  case  was 
TPM  monitoring  identified  as  an  agreed-to  requirement. 

The  function  of  the  SEMP  was  addressed.  Normally  the  FSED 
RFP  issued  by  the  PMO  requires  an  engineering  management  plan, 
which  may  or  may  not  be  specifically  called  a  SEMP.  It  will, 
however,  require  the  contractor  to  detail  in  his  proposal  how  he 
will  perform  the  systems  engineering  functions.  Although  this 
plan  is  reviewed  by  the  government,  during  the  source  selection 
process  it  is  not  consistently  incorporated  into  the  contract 
and  updates  are  not  generally  required.  Although  it  seems 
logical  that  the  engineering  plan  be  kept  updated,  (since  it 
lays  out  how  the  contractor  proposes  to  control  his  engineering, 
design  effort)  this  was  not  found  to  be  the  case.  The  degree  of 


formality  of  the  SEMP  and  the  updating  requirements  seemed  to  be 
associated  with  the  specific  procuring  activity  and  differed 
even  within  each  DoD  agency. 


There  was  general  agreement  that  a  comprehensive  engineering 
management  plan  was  important  for  FSED.  The  consensus  was  that 
the  contractors  proposed  SEMP  should  not  become  a  part  of  the 
FSED  contract.  This  approach  allows  for  flexibility  to  react  to 
unanticipated  needs  without  requiring  a  formal  contractual 
change.  It  was  also  felt  that  formal  update  of  the  engineering 
plan  should  only  be  accomplished  when  there  are  major  changes  in 
the  program;  such  as  a  significant  modification  program  or  major 
schedule  changes  due  to  budget  considerations  which  will  impact 
the  design  time  period,  number  of  test  articles  available,  or 
time  available  for  test. 

Some  concern  was  raised  over  insufficient  evaluation  empha¬ 
sis  being  placed  on  system  engineering  during  the  source  selec¬ 
tion  process.  Specifically,  it  was  felt  that  the  engineering 
management  process  is  not  adequately  analyzed  to  ensure  exis¬ 
tence  of  an  appropriate  task  allocation  system;  that  the  con¬ 
tractor's  process  of  defining  and  measuring  technical  progress 
was  cost  effective,  and  identified  unambiguous,  measurable 
tasks.  Although  there  is  policy  guidance  on  engineering  manage¬ 
ment,  there  are  a  lack  of  definitive  guides  to  aid  evaluation 
boards.  The  perception  is  that  each  program  has  been  unique  in 
terms  of  TPM  definition  and  implementation  plans. 

4.2.2  Parameter  Selection  and  Required  Reporting 


While  the  system  specifications  form  the  logical  basis  for 
the  parameters  that  are  selected  for  monitoring,  the  actual 
selection  process  varied  considerably  from  program  to  program. 
However,  some  consistent  factors  were  clear.  If  the  program 
does  not  push  the  technological  state  of  the  art,  little  concern 
was  evident  in  parameter  monitoring.  Primary  management  empha¬ 
sis  was  placed  on  cost  and  schedule.  Conversely,  if  a  specific 
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requirement  pushed  the  technological  state  of  the  art,  this  was 
flagged  for  monitoring  and  normally  was  called  out  as  a  specific 
parameter. 

When  government  engineers  were  asked  how  they  selected  the 
parameters  that  they  monitored  to  determine  technical  progress, 
the  answers  were: 

o  operational  command  need  -  threat  change 
o  areas  of  risk  -  new  technology,  tight  schedule 
o  cost  drivers  -  high  cost  components 

o  negotiations  with  contractor  -  contractor  recommendations 
o  contract  with  higher  headquarters  -  higher  headquarters 
desires 

In  general,  government  engineers  attempt  to  select  parameters  at 
high  levels  that  cut  across  subsystems  and  are  the  primary  risk 
areas  based  on  analysis  of  the  operational  need. 

In  the  reporting  area,  standard  DIDs  based  on  the  applicable 
MI L— STDS  were  required  by  the  functional  managers  for  technical 
monitoring.  The  requirement  was  based  on  the  particular  disci¬ 
pline,  experience,  and  judgment  of  the  engineer  and  the  expected 
cost  of  the  data.  Several  innovative  PMO's  were  receiving 
contractor  in-house  reports  which  monitored  TPM  rather  than 
requiring  specially  formatted  reports.  This  was  contracted 
through  the  Engineering  Management  Report  (Dl-M-30417 ) .  In  one 
case,  the  contractor's  report  monitored  a  particular  parameter 
down  to  the  component  level  (Ref  2.3.4,  Figure  4).  Another 
contractor  was  monitoring  14  specific  parameters,  developing  27 
separate  charts  and  providing  them  to  the  government  on  a  month¬ 
ly  basis.  Figure  27  provides  an  example  of  how  a  contractor  was 
tracking  and  reporting  the  weight  parameter  at  the  system  lev¬ 
el.  The  chart  clearly  indicates  that  the  specification  alloca¬ 
tion  was  increased  in  April  1983  due  to  the  addition  of  a  func¬ 
tion  to  the  system.  The  written  status  report  that  accompanied 
the  chart  stated  that  current  weight  was  367  pounds  below 


specification  allocation  and  that  the  actual  weightings  of  Line 
Replaceable  Units  (LRUs)  represented  56%  of  the  total  weight 
with  the  remainder  based  on  analysis.  Figure  28  provides  an¬ 
other  sample  of  the  monitoring  of  weight.  The  Interface  Control 
Document  (ICD)  provides  a  not-to-exceed  weight. 

The  PMO's  surveyed  were  developing  technical  performance 
reports  for  higher  headquarters  that  met  specific  format  re¬ 
quirements  levied  upon  them.  They  were  not  requiring  the  con¬ 
tractor  to  comply  with  a  prescribed  format,  but  were  abstracting 
the  data  from  the  contractor  reports.  Figure  29  provides  an 
example  of  an  Air  Force  PMO-required  TPM  chart  for  reporting  at 
Secretarial  reviews. 

4.2.3  Relationship  Between  TPM  and  the  CPR 

In  responding  to  the  question,  "what  is  the  relationship 
between  TPM  and  CPR?"  government  engineers  stated  that  they  were 
not  involved  in  CPR  analysis.  In  only  one  case  did  government 
engineers  analyze  CPR  data.  CPR  data  was  analyzed  by  the  busi¬ 
ness  office,  program  control,  cost  analysis  or  the  procurement 
function  and  one  of  these  offices  was  responsible  for  informing 
the  PM  on  the  results.  Engineering's  perception  of  the  CPR  was 
that  it  was  received  too  late  to  be  a  useful  tool  in  the  daily 
management  of  technical  progress  since  it  identified  problems 
after  the  fact  and  was  not  predictive.  They  did  state  that  the 
PM  reviewed  data  at  monthly  program  status  briefings,  but  that 
the  other  functional  areas  were  responsible  for  the  content,  not 
engineering. 


When  asked  if  there  was  a  link  between  TPM  and  the  CPR, 
government  engineers  agreed  that  a  link  did  exist.  The  link  was 
the  PM  who  received  technical  performance  data  from  the  engi¬ 
neers  and  CPR  (cost  and  schedule)  data  from  other  functional 
areas  at  program  status  reviews.  Generally,  the  PM  would  ask 
questions  concerning  either  TPM  or  the  CPR  and  would  expect  to 
receive  verification  of  a  problem  noted  in  one  system  by  the 
other . 
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Figure  29.  Technical  Status  Report  PMO  to  HQ. 


4.2.4  Contractual  Implications 


The  service  engineers  were  asked  questions  concerning  the 
exchange  of  technical  information  under  various  contract  types 
and  the  importance  of  incentives/award  fees  to  a  TPM  effort. 

Contract  type  was  not  viewed  as  an  inhibitor  to  the  flow  of 
technical  information  between  engineering  communities.  The 
relat_onship  is  open  and  does  not  change  based  on  contract  type, 
although  this  is  not  necessarily  true  for  cost  data.  Under  a 
fixed  price  contract,  the  contractor  may  become  less  responsive 
to  cost  exercises  if  a  benefit  is  not  evident,  while  on  a  cost 
plus  contract,  the  contractor  will  normally  respond  to  any  type 
of  request. 

Some  confusion  existed  when  discussing  incentives  and  award 
fees  for  technical  performance.  Incentives  are  used  to  reward 
technical  performance,  they  require  measurement  and  technical 
performance  is  measurable.  Award  fees  are  a  subjective  reward 
for  contract  performance,  but  not  specifically  technical  per¬ 
formance.  In  discussing  incentives,  the  major  positive  aspect 
is  that  it  clearly  identifies  to  the  contractor  what  the  PM 
considers  important.  An  example  of  an  incentive  on  technical 
performance  is  the  added  fee  for  satellite  operation  beyond  a 
specified  time.  If  the  contractual  agreement  is  for  12  months 
of  operation,  a  negative  incentive  can  be  employed  for  less  than 
12  months  of  operation,  while  a  positive  incentive  for  operation 
beyond  12  months  up  to  some  specified  time  provides  an  added  fee 
to  the  contractor.  Government  engineers  generally  stated  that 
engineers  are  incentivized  by  the  design  challenge  to  success¬ 
fully  complete  a  task  rather  than  contract  incentives.  The 
reason  was  that  when  an  incentive  is  applied  to  a  specific 
parameter,  the  outcome  or  impact  upon  other  parameters  is  not 
necessarily  known.  Emphasis  on  one  area  may  cause  the  contrac¬ 
tor  to  deemphasize  something  else.  This  was  viewed  as  the  major 
negative  effect  of  incentive  type  contracts  for  technical  per¬ 
formance.  If  incentives  are  used  for  technical  performance,  a 
strong  system  engineering  program  is  required. 
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4.3  CONTRACTOR  POINT  OF  VIEW 


4.3.1  Measurement  of  Technical  Performance 

In  an  effort  to  understand  how  contractors  measure  their 
technical  performance  three  areas  were  investigated:  how  they 
planned  their  engineering,  how  tasks  were  assigned,  and  finally 
how  they  measured  their  progress.  On  any  individual  contract 
the  entire  process  starts  with  the  RFP.  Upon  receipt  of  an  RFP 
a  proposal  manager  is  selected  and  a  team  established.  The 
establishment  of  the  proposal  team  is  the  most  common  practice; 
however,  in  companies  that  are  projectized  the  RFP  is  assigned 
to  an  existing  project  group  and  no  special  proposal  team  is 
established.  This  occurs  frequently  with  on  going  programs 
where  the  company  has  been  working  the  specific  project  for 
sometime  and  must  prepare  a  proposal  for  a  follow-on  contract 
( FSD  after  completion  of  D/V). 

In  response  to  the  RFP,  contractors  vary  considerably  in  the 
level  of  detail  planning  that  is  completed  during  proposal 
preparation.  Once  a  corporate  decision  is  made  to  propose, 
whatever  effort  is  viewed  as  required  to  win  the  contract  will 
be  expended.  The  level  of  detail  planning  completed  for  the 
proposal  will  determine  how  quickly  a  contractor  can  start  after 
contract  award.  Generally,  contractors  will  take  the  tasks  from 
the  SOW  and  break  them  down  functionally  in  their  proposal 
preparation;  however,  they  may  or  may  not  expand  the  WBS  and 
develop  cost  accounts.  After  contract  award  a  CWBS  is  developed 
to  some  level  well  below  that  provided  by  the  government  in  the 
RFP.  A  cost  account  plan  is  developed  and  task  oriented  work 
packages  are  established  which  have  measurable  milestones .  The 
schedule  is  reviewed  and  adjusted  as  appropriate.  A  negotiation 
process  occurs  between  the  contractor  PM  and  the  functional 
managers  based  on  labor-hours  and  need  dates.  The  cost  account 
managers  allocate  budget  and  time  to  the  work  packages  based  on 
the  results  of  the  negotiations. 


In  the  early  phase  of  design,  before  there  are  testable 
components,  technical  progress  is  measured  by  two  means:  work 

package  milestone  completion  and  technical  in-house  meetings. 
Completion  of  measurable  work  package  milestones  provided  a 
ready  means  of  identifying  progress.  The  completion  was  veri¬ 
fied  by  the  supervisor  and,  in  some  cases,  by  a  system  engineer¬ 
ing  group.  A  series  of  internal  meetings  —  design  reviews, 
peer  reviews,  interface  meetings,  configuration  meetings  and 
management  reviews  provide  a  status  on  technical  progress. 

These  meetings  occur  at  a  frequency  that  ranges  from  daily  to 
monthly  and  at  locations  varying  from  the  design  room  to  the 
General  Manager's  Conference  Room.  The  attendees  of  these 

meetings  are  predicated  on  the  problem  and  the  level  of  review. 

Frequently,  there  are  daily  meetings  between  both  specialty  area 
and  system  engineering  group  supervisors  and  design  teams  to 
discuss  specific  progress,  plans  and  alternatives. 


4.3.2  Parameter  Selection,  Problem  Identification  and  Correc¬ 


tive  Action 


Contractors  consistently  stated  that  the  government 
established  the  key  high  level  parameters  and  these  were  identi¬ 
fied  in  the  SOW.  Their  functional  engineers  review  the  SOW  and 
system  specifications  during  their  proposal  preparation  and  make 
sure  that  the  specifications  make  sense  and  that  in  their  judge¬ 
ment,  the  key  parameters  the  government  plans  to  monitor  are  the 
right  ones.  Recommended  changes  may  be  included  in  the  proposal 
package  or  negotiated  after  contract  award.  This  SOW  and  speci¬ 
fication  analysis  forms  the  basis  of  the  contract  negotiations. 


Problems  impacting  system  technical  performance  are  identi¬ 
fied  by  various  tracking  systems.  At  the  completion  of  each 
component  design  iteration,  various  design  and  design  support 
disciplines  review  the  design  based  on  their  understanding  of 
the  allocated  requirements  and  the  impact  upon  their  function. 
Weight  engineers  estimate  the  weight  to  verify  the  expected 


weight;  thermal  engineers  evaluate  the  design  based  on  the  rated 
temperature  range;  electronic  engineers  evaluate  the  suscepti¬ 
bility  to  electromagnetic  interference  (EMI)  either  from  itself, 
nearby  components,  or  other  systems.  Structural,  guidance, 
aerodynamic,  reliability,  producibility ,  maintainability,  human 
engineering  all  evaluate  the  proposed  design  based  on  their 
requirements  and  in  accordance  with  the  SEMP  procedures. 

For  example,  in  a  recent  design  iteration  of  an  Auxiliary 
Power  Unit  (APU)  for  an  advanced  fighter,  it  was  discovered  that 
the  hot  exhaust  from  the  APU  would  blow  into  the  avionics  bay 
and  on  the  maintenance  person  servicing  the  avionic,  equipment. 
Since  the  APU  had  to  be  on  during  maintenance  of  the  avionics 
equipment,  a  redesign  of  the  APU  exhaust  system  was  required. 
This  problem  could  have  been  identified  by  any  number  of  spe¬ 
cialties,  i.e.,  safety,  maintenance,  avionics  engineering,  or 
human  engineering  but  became  clear  only  when  the  component  was 
viewed  from  the  systems  perspective.  Each  system  engineering 
discipline  views  the  design  or  test  based  on  their  experience 
and  the  requirement. 

Monitoring  of  the  completion  of  work  package  milestones  may 
also  flag  problems.  When  a  work  package  is  running  behind 
schedule,  several  situations  can  exist.  It  may  be  that  the 
estimate  to  complete  the  effort  was  wrong  and  no  problem  exists 
other  than  determining  what  impact  the  late  completion  will  have 
on  other  activities,  or  a  real  problem  may  exist  and  more  re¬ 
sources  must  be  expended  or  an  alternative  design  must  be  devel¬ 
oped. 

In  most  cases,  no  matter  who  identifies  the  problem,  some 
form  of  in-house  report  is  developed  to  document  it.  This 
report  is  provided  to  various  levels  of  management  to  keep  them 
informed  and  to  ensure  that  the  appropriate  resources  are  ap¬ 
plied  in  developing  the  fix.  These  reports  are  called  by  vari¬ 
ous  names,  such  as  trouble  reports,  red  darters,  investigation 


reports,  technical  performance  reports,  or  daily  interest 
items.  In  some  cases,  contractors  have  provided  these  in-house 
reports  to  the  government  as  a  part  of  their  TPM  program. 

Managers  of  the  various  disciplines  evaluate  these  reports 
and  develop  impact  statements,  workarounds  and  corrective  ac¬ 
tions  based  upon  the  impact  on  their  specific  schedule.  A 
problem  which  causes  a  delay  on  the  critical  path  will  have  a 
ripple  effect  upon  the  entire  program.  For  this  reason,  correc¬ 
tive  action  is  usually  attempted  at  the  lowest  level  possible  so 
as  to  have  minimum  effect  upon  the  next  higher  level;  i.e., 
correct  at  component  level  rather  than  subsystem  level.  Daily 
meetings  between  various  groups,  matrix  engineers  and  functional 
departments,  subsystem  managers,  and  design  engineers  are  held 
to  determine  system  status  and  address  problems.  The  makeup, 
frequency,  and  decision  authority  at  these  meetings  is  the 
result  of  the  company's  system  engineering  process. 

As  problems  occur  which  impact  key  parameters,  formal  trade¬ 
off  studies  are  developed  and  reviewed  by  the  system  engineer 
and  project  manager.  This  is  part  of  the  design  iteration 
process.  Initial  requirements  allocation  to  subsystems  or 
components  may  have  to  be  changed  and  reallocation  made.  This 
is  the  challenging  part  of  system  engineering,  keeping  all 
phases  of  the  design  open  so  that  trade-offs  can  be  made.  As 
the  design  freezes  in  various  subsystems,  the  ability  to 
reallocate  decreases,  yet  design  progress  schedules  are  based  on 
the  timely  freezing  of  each  component  design. 

4.3.3  Relationship  Between  TPM  and  CPR 

The  contractor  engineering  community  was  totally  conversant 
with  the  concept  of  C/SCSC  and  the  CPR  report.  The  earned  value 
method  of  measuring  performance  was  completely  ingrained  within 
the  companies. 


The  CPR  is  the  foundation  of  the  contractor  Management 
Information  System  (MIS)  which  provides  management  program 
status.  Cost  and  schedule  data  is  reviewed  at  the  cost  account 
level  anywhere  from  weekly  to  monthly.  The  availability  of  CPR 
data  is  not  considered  a  problem  by  the  contractors.  Labor  hour 
data  is  available  very  quickly,  and  is  used  as  a  status  indica¬ 
tor.  Excessive  hours  frequently  correlates  with  a  technical 
problem. 

When  responding  to  the  question;  "if  C/SCSC  and  the  CPR  were 
no  longer  required,  how  would  you  manage?"  The  answer  in  almost 
every  case  was  by  some  form  of  earned  value  system.  The  con¬ 
tractors  indicated  they  would  still  develop  cost  accounts,  have 
task-oriented  work  packages,  and  develop  milestone  schedules. 
"We  would  still  negotiate  budget  to  the  cost  account  level  and 
allocate  to  work  packages".  These  answers  were  fortified  by  the 
fact  that  CPR's  were  developed  for  firm  fixed  priced  ( FFP ) 
contracts  even  though  not  required  by  the  government.  One 
contractor,  which  has  80%  of  its  business  FFP,  creates  an  in- 
house  cost  performance  report  on  all  contracts. 

The  consensus  of  the  industry  engineering  community  was  that 
C/SCSC  and  the  SEMP  require  a  degree  of  advanced  planning  that 
is  essential  to  good  program  and  engineering  management. 

In  responding  to  the  question:  "Is  there  a  link  between  the 
CPR  and  TPM;"  the  engineering  community  identified  their  PM  or 
chief  engineer  as  the  link.  It  is  these  individuals  who  evalu¬ 
ate  the  CPR  and  TPM  variance  at  formal  in-house  status  reviews. 
These  reviews,  much  like  the  governmental  program  review,  pro¬ 
vide  the  PM  a  review  of  technical  progress,  cost  and  schedule 
status.  Also  identified  as  a  link  between  the  CPR  and  TPM  was 
the  accomplishment  of  measurable  milestones  established  by  the 
contractor.  The  milestone  schedule  is  an  integral  part  of  both 
systems  and  therefore  forms  a  link  between  the  two  systems. 


The  area  of  contractual  relations  and  technical  performance 
was  evaluated  in  terms  of  data  availability,  incentives  and 
award  fees. 

The  type  of  contract  did  not,  in  the  contractor's  view, 
inhibit  the  flow  of  technical  data  between,  the  contractor  and 
the  service  program  office.  When  the  government  program  office 
put  an  engineer  in  the  plant,  he  was  invited  to  the  in-house 
program  meetings  and  could  report  what  he  wanted  back  to  the 
program  manager.  These  in-house  meetings  were  in  many  cases  the 
place  where  problems  were  addressed  and  solutions  freely  debated. 

An  element  of  contradiction  existed  in  the  area  of  effec¬ 
tiveness  of  award  fees  and  incentives  for  technical  perform¬ 
ance.  Several  contractors  stated  that  award  fees  had  been  used 
for  technical  performance  in  the  past.  They  further  stated  that 
the  subjectiveness  of  the  award  fee  required  the  government  to 
accomplish  considerable  planning  which  was  not  always  done 
thoroughly.  Award  fees  are  not  generally  appropriate  devices 
for  assuring  superior  technical  performance. 

Contractually  specified  incentives  did  identify  to  the 
contractor  what  the  government  PM  considers  most  important. 
Concern  was  raised  over  their  use  for  technical  performance 
parameters  because  it  is  difficult  to  predict  the  system  level 
effect.  For  example,  the  designer  of  an  ordnance  system  may  be 
provided  an  incentive  to  reduce  weapon  reload  time.  His  solu¬ 
tion  might  be  the  incorporation  of  an  unobstructed  trunk  between 
the  weapon  launcher  and  the  magazine.  This  may  reduce  weapon 
reload  time;  however,  it  could  degrade  system  water-tight  integ¬ 
rity  and  combat  survivability  or  maintainability.  In  addition, 
the  contractors  stated  that  the  inflexible  nature  of  incentives 
tended  to  hamper  their  ability  to  make  objective  system  level 
trade-offs. 


Information  on  incentives  and  award  fees  does  not  appear  to 
flow  down  into  the  contractor  organization.  The  design  engineer 
does  not  necessarily  know  if  there  is  an' incentive  on  the  con¬ 
tract  or  of  progress  toward  its  achievement.  A  tightening  of 
allocated  specifications  is  generally  how  management  ensures 
that  the  incentive  is  earned.  Some  publicity  and  verbal  exchange 
occurs  on  incentives  and  award  fees  especially  in  the  area  of 
design  to  cost;  however,  little  is  done  to  publicize  incentives 
associated  with  technical  performance  design  features. 

4.4  SURVEY  SUMMARY 


The  government  engineer's  function  is  to  develop  the  system 
specifications  and  the  statement  of  work,  select  the  contractors 
and  monitor  their  technical  progress,  all  based  on  the  mission 
requirement.  The  contractor  engineer  must  utilize  the  system 
specification  and  statement  of  work,  develop  a  proposal,  win  the 
contract,  and  develop  the  weapon  system  based  on  the  specifica¬ 
tions  . 


The  survey  revealed  agreement  between  the  two  engineering 
communities  in  the  areas  of  contract  incentive/award  fee  effec¬ 
tiveness,  information  flow,  and  linkage  between  the  CPR  and  TPM. 
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Some  confusion  existed  about  the  appropriateness  of  award 
fees  for  technical  performance.  Since,  by  its  nature,  technical 
performance  should  be  measurable,  subjective  award  fees  do  not 
appear  to  be  the  appropriate  vehicle  to  incentivize  technical 
performance.  All  agreed  the  incentives  for  technical  perform¬ 
ance  need  to  be  carefully  structured  because  they  can  cause 
unexpected  results.  Contract  type  does  not  inhibit  the  flow  of 
technical  information.  Both  engineering  communities  identified 
the  PM  as  the  only  existing  effective  link  between  the  CPR  and 
TPM.  The  contractors  elaborated  by  also  listing  the  accomplish¬ 
ment  of  common  measurable  milestones  as  a  link. 
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Several  survey  questions  identified  different  perceptions 
among  the  engineering  communities  in  the  areas  of  SEMP,  CPR  and 
involvement  in  program  management.  DoD  PMO  engineers  appeared 
to  be  ambivalent  about  the  SEMP,  some  placing  little  value  on  it 
as  a  requirement  and  having  little  concern  over  whether  it  was 
updated.  On  the  other  hand,  contractor  engineers  stated  that 
this  document  was  their  engineering  management  baseline  by  which 
they  established  their  plan  for  accomplishing  the  essential 
system  engineering  functions  of  design. 

The  CPR  forms  the  basis  for  the  contractor's  MIS,  and  is 
monitored  by  the  engineering  community.  TPM  and  the  CPR  are 
briefed  by  the  contractor  system  engineers  to  the  PM  and  are 
tracked  together.  Work  package  completion  is  utilized  to  moni¬ 
tor  design  group  technical  progress.  The  government  engineers 
monitor  technical  progress,  but  have  little  involvement  with  the 
CPR.  The  CPR  is  monitored  by  a  totally  separate  organization. 

Micro  management  by  the  service  engineer  was  not  listed  as  a 
concern  by  the  contractors;  however,  when  DoD  engineers  were 
asked  the  question,  "What  is  your  decision  organizations  process 
when  variance  occurs  in  cost,  schedule  or  technical  perform¬ 
ance?"  the  answers  covered  the  entire  spectrum  of  involvement. 
One  answer  was  —  "It's  the  contractor's  problem,  not  ours"  — 
total  non-involvement.  Another  responded  with  —  "set  up  a 
tiger  team,  go  to  the  contractor's  plant  and  work  the  problem" 

possible  over-involvement.  Neither  of  these  respondents 
stated  that  they  would  first  make  an  effort  to  fully  understand 
the  problem  and  the  contractor's  corrective  action  plan. 
Monitoring  requires  thorough  evaluation  of  the  corrective  action 
plans  and  the  availability  of  sufficiently  detailed  timely 
information.  When  the  contractors  planned  solution  will  impact 
other  functions,  the  service  engineer  must  become  involved.  If 
the  problem  is  such  that  only  the  PMO  can  solve  it  through  a 
change  in  specification,  relief  on  schedule,  or  a  change  in  the 
budget,  then  the  PMO  must  take  the  initiative  to  implement 
appropriate  contract  change  proposal  action. 
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One  of  the  most  significant  results  of  the  survey  was  the 
determination  that  an  adversarial  relationship  does  not  exist 
between  the  engineering  communities.  Contractor  engineers  tend 
to  be  open  with  their  customers  and  provide  technical  informa¬ 
tion  regardless  of  contract  type.  In  addition,  the  use  of  the 
CPR  by  the  contractor  as  the  foundation  of  the  program  Manage¬ 
ment  Information  System  (MIS)  validates  that  it  is  an  estab¬ 
lished  system  accepted  by  the  engineering  community.  This  is 
not  the  case  for  the  government  engineering  community. 
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CHAPTER  5 


IMPLEMENTATION  OF  A  TPM  PROGRAM 

5.1  INTRODUCTION 

This  chapter  is  designed  to  review  the  actions  the  Program 
Office  can  take  prior  to  and  after  the  first  FSD  contract  award 
in  order  to  have  a  viable  cost  effective  TPM  Program.  The 
chapter  concludes  with  some  impacts  technology  is  having  on  TPM 
implementation. 

TPM  is  a  means  of  keeping  the  government  PM  informed  on  the 
status  of  his  contractor's  technical  progress.  It  is  the  aggre¬ 
gation  of  technical  information  in  a  manner  suited  to  prompt 
decision  making.  The  PM  cannot  be  expected  to  be  an  expert  in 
all  engineering  disciplines;  TPM  should  be  an  integral  part  of 
the  management  review  process,  allowing  management  by  excep¬ 
tion.  This  approach  enables  the  PM  to  establish  control  limits 
for  the  value  of  each  significant  parameter  and  to  concentrate 
attention  on  those  parameters  that  presently  or  eventually  will 
fall  outside  the  control  limits.  It  allows  the  PM  to  direct 
efforts  toward  the  design  aspects  that  have  a  major  influence  on 
the  outcome  of  the  program.  Complex  programs  involving  the 
interactions  of  many  parameters  can  be  effectively  managed  by 
the  prompt  selective  application  of  management  resources  where 
the  benefit  is  the  greatest,  if  through  TPM  these  areas  can  be 
readily  identified. 

Implementation  of  a  TPM  program  and  the  application  of 
C/SCSC  both  require  careful  front  end  planning  by  the  contrac¬ 
tor.  The  contractors  SEMP  will  provide  information  on  how  he 
plans  to  conduct  system  engineering.  The  on-site  review  of  the 
system  will  provide  confidence  and  understanding  which  will  aid 
in  evaluating  problems  in  the  future.  The  PM's  review  of  TPM 
reports  and  the  CPR  will  be  the  primary  link  between  the  two 
systems. 


5.2  UPFRONT  ACTIVITIES 


The  successful  Program  Manager  must'  carefully  direct  the 
development  of  the  FSD  RFP.  The  RFP  is  the  need  document  to 
which  the  contractors  will  respond,  therefore  it  is  perhaps  the 
most  important  document  the  PM  will  issue  in  his  tenure.  The 
SOW  and  system  specifications  must  unambiguously  state  the 

program  objectives.  The  SOW  should  identify  a  carefully  tai¬ 
lored  TPM  Directive  as  a  requirement  or  a  guide  (Air  Force 
MIL-STD-499A,  Army  FM770-78).  A  formal  TPM  process,  tailored  to 
the  particular  program  needs,  should  be  required  in  the  RFP, 
allowing  for  incremental  assessment  of  design  attainment  during 
the  periods  between  the  MIL-STD-1521A  reviews  ( PDR,  CDR,  etc). 
The  PM  should,  in  the  RFP,  define  the  schedule,  depth  and  docu¬ 
mentation  of  program  status  reviews  desired.  This  will  depend 
on  a  number  of  factors,  among  these  are:  program  visi-  bility, 
risk  and  management  complexity.  As  an  example,  there  may  be 

three  levels  of  status  meetings:  First,  monthly  techni-  cal 

interchange  meetings  between  the  government  senior  engineer-  ing 
staff  and  their  counterparts;  second,  bimonthly/quarterly 
program  manager  meetings  between  the  PM  and  senior  staff  with 

the  contractor's  counterparts;  and  finally,  quarterly/semiannual 
senior  management  reviews  between  the  PM's  commander  and  the 
CEO.  Each  of  these  revie  rs  should  be  structured  to  assess  the 
status  of  the  program  at  different  detail  levels,  therefore  the 
agenda  should  be  so  structured.  The  frequency  and  formats  of 

reports  required  from  the  contractor  for  TPM  monitoring  should 
be  carefully  structured. 

5.2.1  Parameter  Selection 


rs: 


Increasing  complexity  of  programs  demand  the  tracking  of 
multiple  TPM  parameters.  Experience  suggests  that  specific 
parameters,  and  the  number  of  parameters  to  be  tracked  will 
depend  on  the  characteristics  of  the  program,  the  selected 
acquisition  strategy,  and  recognized  risk  areas.  Clearly,  the 
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level  of  new  technology  embedded  in  the  program  vice  off-the- 
shelf  or  existing  technology  is  a  critical  factor  in  the  process 
in  order  to  logically  select  TPM  parameters. 

The  number  of  parameters  appropriate  for  a  program  may  vary 
widely.  Some  programs  have  selected  five  to  ten  critical  system 
level  parameters.  In  certain  cases,  additional  parameters  have 
been  specified  for  subsystems  or  lower  level  components.  In  any 
event,  it  is  increasingly  apparent  that  parameters  selected  to 
be  worth  the  cost  should  be  relatively  independent  of  each 
other,  and  significant  indicators  of  system  capability  relative 
to  overall  objectives.  Where  parameters  are  dependent,  there 
may  be  cases  where  contradictory  indications  are  provided  to 
program  management.  The  ability  to  use  these  parameters  to 
guide  management  decisions  and  develop  trade-offs  is  degraded, 
and  prompt  management  response  inhibited. 

As  an  example,  if  three  parameters  are  recognized  as  being 
critical,  and  two  of  these  parameters  are  related  to,  or  aggre¬ 
gate  to  provide  the  third,  it  is  possible  that  the  top  level 
parameter  will  be  satisfied  while  one  of  the  other  factors  is 
not  satisfied.  To  further  amplify  this  case,  if  in  the  develop¬ 
ment  of  a  guided  artillery  round,  single  shot  kill  probability, 
single  round  reliability  and  accuracy  are  selected  as  three 
parameters,  the  following  situation  could  occur.  The  probabil¬ 
ity  of  a  single  shot  kill  (PSSK)  is  a  function  of  reliability 
and  accuracy.  Accuracy  could  be  specified  as  X  meters.  Relia¬ 
bility  specified  as  R,  and  the  resultant  probability  of  a  single 
shot  kill  as  Y.  In  testing,  system  accuracy  is  measured  as  1/2 
X  meters  or  better  than  the  specified  parameter,  reliability  is 
R'  which  is  less  than  the  specified  level  of  R.  The  calculated 
single  shot  kill  probability  is,  therefore,  Y'  which  may  be 
greater  than  the  specified  Y.  The  program  could  encounter 
delays  in  entering  rate  production  because  it  has  not  met  its 
stated  reliability  objective. 


Presuming  that  the  calculation  is  correct,  fewer  shots  will 
be  required  per  target,  thereby  reflecting  an  increase  in  tar¬ 
geting  effectiveness,  and  other  benefits  that  could  include 
reduction  of  exposure  of  the  launch  crew  to  enemy  targeting,  and 
also  more  kills  per  crew  in  unit  time.  On  the  other  hand,  the 
reduced  reliability  can  equate  to  increased  maintenance  and 
support  requirements,  in  the  case  of  a  maintainable  round  and  in 
the  possibility  of  round  "jams." 

A  simple  answer  is  not  always  the  right  answer.  The  ques¬ 
tion  is,  what  is  important  about  an  artillery  round?  Possibly, 
the  key  parameter,  in  this  case  of  a  ’wooden*  or  nonmaintainable 
round,  should  have  been  the  single  shot  kill  probability  rather 
than  three  interrelated  parameters.  The  net  effect  would  have 

been  no  program  delay  and  reduced  development  cost.  The  con- 

* 

tractor,  therefore,  would  have  had  the  ability  to  take  the 
single  parameter  of  a  probability  of  a  single  shot  kill  and 
trade  off  between  the  various  levels  of  reliability  and  accu¬ 
racy.  Specifically,  calling  out  all  three  interrelated  parame¬ 
ters  as  specification  values  without  providing  additional  guid¬ 
ance  as  to  priority,  could  have  degraded  the  contractor's  abili¬ 
ty  to  trade-off  design  and  program  alternatives. 

For  Example: 

PSSK  =  Probability  of  a  single  shot  kill 

X  =  Accuracy  -  measured  by  the  number  of  fired 
rounds  impacting  within  lethal  distance 

R  =  Reliability  -  measured  by  the  number  of 
delivered  rounds  that  fire 

PSSK  =  X  X  R 

If  a  PSSK  of  .76  is  required  then  .76  =  .90  x  .85 

and  R  can  be  varied  to  produce  the  same  PSSK 
.76  =  .95  x  .80  or  .88  x  .87 

Although  it  is  usually  desirable  to  keep  TPM  parameters  at 
the  system  level  it  will  not  always  be  practical.  If  a  program 
is  organized  into  separate  segments  and  the  Program  Office 


rather  than  a  prime  contractor  has  responsibility  for  integra¬ 
tion,  lower  level  parameters  may  have  to  be  defined  that  are 
significant  for  each  segment.  For  example,  on  an  aircraft 
program  where  speed  is  a  key  parameter,  the  Program  Office  may 
have  to  allocate  weight  to  the  various  system  contractors,  and 
thrust/weight  factors  to  an  engine  contractor. 

State-of-the-Art  technology  may  be  the  key  to  criteria 
guiding  the  selection  of  parameters.  The  risk  implicit  in 
development  of  a  system  and  the  impact  on  the  mission  of 
"missed"  objectives  will  drive  the  specified  level  for  a  parame¬ 
ter  to  a  high  level,  and  possibly,  more  importantly,  to  a  level 
that  cannot  be  reduced  without  degrading  the  capability  of  the 
system  to  perform  its  mission.  A  radar  detection  range  that 
degrades  below  a  certain  level  may  reduce  the  pilot's  response 
time  to  launch  a  missile  to  achieve  a  kill  at  an  acceptable  or 
safe  range.  However,  a  reliability  degradation  in  the  radar 
from  a  "high"  level  to  a  slightly  lower  level  may  not  signifi¬ 
cantly  degrade  mission  effectiveness,  and  may  only  slightly 
increase  its  support  cost.  The  net  impact  on  Life  Cycle  Cost 
( LCC )  may  be  nominal,  unless  the  attempt  to  capture  the  speci¬ 
fied  reliability  forces  the  program  into  a  complex  "fix"  or  a 
technology  enhancement  effort  that  could  significantly  increase 
development  cost  and,  therefore,  LCC.  Where  the  possible  impact 
on  program  schedule  is  considered,  logic  may  drive  this  program 
to  accept  the  "fact  of  life". 

The  government  PM,  chief  engineer  and  system  engineering 
chiefs  from  the  key  functional  disciplines  should  meet  and  make 
the  final  decision  on  what  parameters  form  the  foundation  of  the 
program  and  will  be  monitored  in  the  TPM  program. 

5.2.2  System  Engineering  Management  Requirements 

The  FSD  RFP  should  require  delivery  of  a  comprehensive 
engineering  management  plan.  The  SEMP  is  one  format,  however, 
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contractors  have  engineering  management  procedures  and  internal 
plans  which  they  should  be  encouraged  to  propose  if  they  satisfy 
the  requirement.  This  plan  must  explain  how  the  contractor  will 
manage  his  integrated  engineering  effort  and  must  become  his 
engineering  management  baseline.  The  documentation  provided  by 
the  contractor  in  the  form  of  TPM  reports  is  equally  important. 
The  data  call  during  RFP  preparation  is  a  very  important  func¬ 
tion  which  frequently  does  not  get  the  visibility  needed  within 
the  PMO.  The  Data  Requirements  Review  Board  (DRRB)  will  deter¬ 
mine  what  information  the  Program  Office  should  receive  from  the 
contractor.  How  this  is  managed  will  impact  the  Program  Of¬ 
fice's  ability  to  effectively  monitor  performance.  The  Program 
Manager  should  chair  this  board.  The  SOW  should  clearly  iden¬ 
tify  the  TPM  parameters  and  data  required  from  the  contractor. 
Some  contractors  have  established  specific  formats  for  TPM 
management  and  have  been  using  them  for  years.  If  they  provide 
the  needed  data,  they  can  be  requested  through  use  of  the  Data 
Accession  List.  The  requirement  for  a  specific  DID  merely 
establishes  the  format  vehicle.  Keep  in  mind  a  piece  of  paper 
never  solved  a  technical  problem  —  you  need  current,  clear 
information  and  this  intent  should  be  specified  in  the  SOW. 
Once  attainment  of  a  TPM  parameter  has  been  reached  and  design 
frozen,  then  discontinuance  of  reporting  should  be  automatic  if 
no  further  activity  can  be  expected. 

5.2.3  Source  Selection  Criteria/Procedures 


The  RFP  has  required  the  contractor  to  develop  a  proposal 
which  states  as  clearly  as  possible  how  the  development  system 
engineering  process  including  TPM  will  be  managed.  The  source 
selection  is  the  next  critical  step  to  a  contract  award.  This 
step  can  be  an  extremely  important  function  for  engineering.  In 
evaluating  the  contractor  response  to  the  SOW,  special  attention 
must  be  placed  on  the  contractor's  engineering  management  pro¬ 
cess,  including  TPM,  and  appropriate  selection  factor  weighting 
should  be  d"  eloped  by  the  PMO.  The  system  engineering  TPM 
proposal  evaluation  should  consider: 


o  Have  the  SOW  tasks  all  been  clearly  allocated  and  are 
they  readily  traceable  to  the  system  specifications? 

o  What  procedures  or  capabilities  exist  for  conducting 
analysis  and  data  aggregation  to  the  specification  level? 

o  Is  the  engineering  milestone  chart  presented  in  suffi¬ 
cient  detail  to  show  the  interrelation  of  design  activi¬ 
ties,  technical  reviews,  analysis  group  reports,  simula¬ 
tion,  and  tests? 

o  Does  system  engineering  control  bring  all  functional 
areas  into  a  balanced  integrated  effort  using  clear  firm 
rules  and  procedures? 

o  Has  a  TPM  plan  been  developed  which  will  be  both  respon¬ 
sive  to  the  government  PM  needs  and  avoid  duplicated 
efforts  and  special  test/analysis  requirements? 


Concern  was  raised  during  the  survey  reported  in  Chapter  4 
that  in  the  past,  insufficient  attention  has  been  paid  to  engi¬ 
neering  management  capability  during  source  selection.  Proposed 
cost  and  schedule  may  have  been  the  driving  factors  in  source 
selection.  What  is  required  is  a  balance  between  cost,  schedule 
and  technical  performance  capabilities  enforced  thru  source 
selection  criteria  weights. 


5.2.4  Negotiation 


A  program  that  was  carefully  laid  out  in  the  RFP  and  re- 
sponsed  to  in  the  Proposal  can  become  disjointed  during  negotia¬ 
tion  if  the  balance  between  cost,  schedule  and  technical  per¬ 
formance  is  disrupted.  The  two-step  procurement  practice  which 
first  evaluates  technical  capability,  placing  all  contractors 
who  qualify  above  some  level  to  equal  and  then  evaluates  cost, 
may  impact  system  engineering  management  and  the  desired  TPM 
program.  A  contractor  who  barely  qualifies  technically  and  has 
a  weak  TPM  program  may  have  the  lowest  cost  and  win  the  con¬ 
tract.  This  may  not  be  the  best  value  for  the  dollar  to  the 
government. 
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Data  reduction,  schedule  changes,  deliverable  adjustments, 
etc.,  should  be  evaluated  by  the  appropriate  functional  disci¬ 
pline  before  a  decision  is  made.  The  effect  of  incentives,  both 
positive  and  negative,  on  key  technical  parameters,  for  improved 
performance,  should  be  evaluated  by  the  PMO  due  to  the  possible 
unexpected  outcomes. 

5.3  AFTER  CONTRACT  AWARD 
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Since  the  SEMP  is  the  way  the  contractor  is  going  to  manage 
the  engineering,  it  is  necessary  that  it  be  updated  to  reflect 
the  results  of  the  negotiations.  An  update  60  days  after  con¬ 
tract  award  should  be  contained  in  the  negotiated  contract. 
This  provides  the  engineering  management  baseline  which  will  be 
tracked  by  both  the  government  Program  Office  and  the  contrac¬ 
tor.  The  baseline  is ‘important  for  the  contractor's  allocation 
and  subsequent  measurement  of  performance. 

With  the  negotiated  contract,  SEMP  update,  and  TPM  plan  in 
hand,  the  important  PMO  task  is  for  the  PM  and  staff  to  learn 
the  contractors  engineering  process  as  it  is  actually  executed. 
The  PM  and  staff  must  visit  the  contractors  facilities  for  a 
reasonable  time  (not  just  a  day  or  two)  to  understand  exactly 
how  the  contractor  accomplishes  the  engineering  activity  if  D/V 
has  not  provided  the  familiarity.  This  will  provide  the  PM  and 
staff  the  understanding  of  the  underlying  organizational  struc¬ 
ture  and  individuals  behind  the  technical  information  provided 
by  the  contractor  at  future  meetings  and  in  TPM  reports.  With¬ 
out  the  PM's  personal  commitment  to  understanding  the  process, 
the  credibility  of  the  contractor's  reports  will  always  be 
suspect.  This  visit  will  enhance  the  PM's  ability  to  evaluate 
future  reports  and  not  engage  in  micro-management  every  time  a 
problem  surfaces. 

The  technical  staff  of  the  PMO,  who  monitor  technical  pro¬ 
gress  through  TPM,  must  also  familiarize  themselves  with  how  the 
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contractors  will  be  working  particular  functional  areas.  The 
technical  staff  must  normally  monitor  progress  at  several  levels 
lower  than  that  monitored  by  the  PM  and  need  a  greater  degree  of 
familiarity  with  the  sources  of  information  they  will  receive. 

As  FSD  begins  review  of  contractor  TPM  status  reports, 
schedules  and  CPR  reports  will  be  the  primary  PMO  means  of 
monitoring  contractor  technical  progress  toward  parameter  a- 
chievement.  Meetings  between  the  government  PM  and  his  staff 
and  the  contractor  should  provide  the  greatest  insight  into 
technical  progress  during  the  design  phase.  Meetings  specified 
in  the  contract  will  be  organized  by  the  contractor.  Design 
status  in  relation  to  the  selected  TPM  parameters  should  be  the 
agenda  framework.  The  CPR  should  be  reviewed  at  the  CWBS  level 
that  best  relates  to  the  TPM  parameters  being  monitored.  The 
two  reporting  systems  should  be  used  to  evaluate  progress;  CPR 
for  cost  and  schedule  and  TPM  reports  for  technical  perform¬ 
ance.  For  example,  in  a  vehicle  program  status  review,  techni¬ 
cal  problems  were  briefed  concerning  the  development  of  the 
transmission.  The  review  of  the  CPR  at  the  CWBS,  level  4,  drive 
train,  indicated  a  schedule  variance.  The  Problem  Analysis  of 
the  CPR,  Format  5,  provided  an  analysis  of  the  schedule  vari¬ 
ance,  identification  of  the  problem  and  planned  corrective 
action.  The  TPM  report  identified  the  problem  and  the  CPR 
confirmed  it.  The  system  impact  and  planned  corrective  actions 
were  reviewed  in  terms  of  the  key  parameter,  reliability.  The 
transmission  problem,  if  not  corrected,  will  impact  the  entire 
program  and  will  prevent  the  mean  time  between  repair  parameter 
from  reaching  its  required  level.  This  provided  a  cross-check 
against  the  technical  problem  and  the  time  extent  of  slippage 
being  experienced. 

Over-involvement  and  its  prevention  will  be  a  problem 
throughout  the  PM's  relationship  with  the  contractor.  The 
majority  of  TPM  data  being  provided  by  the  contractor  is  de¬ 
signed  to  keep  the  government  PM  informed.  Meetings  held  with 
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the  contractor  will  either  be  for  information  or  approval  of  a 
specific  action.  The  government  PM  and  staff  should  be  evaluat¬ 
ing  contractor  design  decisions  based  on  time  and  dollars.  In  a 
properly  managed  program  design  decisions  are  made  by  the  con¬ 
tractor,  however  there  will  be  information  that  indicates  the 
problem  cannot  be  resolved  within  the  contractors  area  of  re¬ 
sponsibility  or  might  be  resolved  more  efficiently  by  others. 
That  is  the  time  for  the  PMO  to  take  action  to  optimize  the 
problem  resolution. 

5.4  CAPITALIZING  ON  AVAILABLE  MANAGEMENT/TECHNOLOGY 

Computer  aided  design,  engineering  and  manufacturing  as  well 
as  networking,  (linking  of  computers,  terminals  etc.)  are  becom¬ 
ing  factors  in  the  cost  and  utility  of  a  TM  program.  Automation 
technology  should  facilitate  TPM.  Accounting  type  functions  are 
automated  and  can  be  accomplished  rapidly.  The  CPR  is  automated 
by  almost  all  contractors.  The  PMO’s  ability  to  monitor  sched¬ 
ules  and  keep  track  of  dollars  can  be  almost  on  a  real  time 
basis.  The  CPR  is  available  to  the  contractor  within  two  weeks 
after  close-out  and  labor  hour  data  is  available  daily.  The  CPR 
should  be  provided  to  the  government  PMO  within  a  maximum  of 

four  weeks  after  close-out. 

Computer  aided  design,  engineering  and  manufacture  (CAD, 
CAE,  CAM)  are  in  use  throughout  industry  and  the  use  is  grow¬ 
ing.  The  degree  of  automation  varies  considerably  by  contrac¬ 
tor.  This  year  one  contractor  interviewed  is  installing  700 
engineering  stations.  Figure  30  provides  a  projection  of  CAD 

and  CAM  use.  As  this  area  develops,  the  product  definition  data 
base  will  improve  in  timeliness  and  assessibility,  this  is 

fundamental  for  a  credible  assessment  of  technical  performance 
progress  at  any  point  in  time.  With  development  of  added  simu¬ 
lations  and  algorithms  the  system  design  data  base  will  provide 
real  time  performance  analysis  capability.  Information  is  now 

feeding  the  contractors  data  base,  but  it  was  not  apparent  that 
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this  is  helping  the  government  PMO.  There  appears  to  be  no 
existing  attempts  to  directly  extract  key  technical  parameter 
information  for  PMO  use,  and  real  time  PMO  analysis  of  the 
design  process  relative  to  TPM  is  still  awaiting  implementation. 

So  where  does  industry  now  stand?  We  have  an  automated  CPR 
system  and  CAD,  CAE,  CAM  systems  in  various  stages  of  implemen¬ 
tation.  An  overall  consolidated  contractor  integrated  MIS  which 
captures  technical  progress  seems  to  be  missing.  The  computer 
power  is  available  for  such  a  system;  however,  the  contractor  as 
well  as  government  personnel  have  not  determined  their  specific 
needs  for  information.  In  almost  every  survey  case  contractors 
are  reviewing  their  MIS.  As  the  program  manager,  you  need  to 
know  your  contractor's  status.  You  will  not  have  an  easy  task 
in  developing  automated  delivery  of  management  information  to 
the  PMO  because  there  is  not  yet  a  single  point  to  tap  into  the 
contractors  systems  for  all  PMO  needs.  This  is  another  reason 
why  the  government  PM  must  understand  how  the  contractor  is 
managing  the  engineering  effort.  Consideration  might  logically 
be  given  to  the  degree  of  automation  compatibility  and  potential 
efficiency  the  contractor  MIS  has  with  the  PMO  during  CEM  or  DV 
source  selection. 

The  next  technological  area  is  electronic  networking.  It  is 
not  uncommon  for  the  government  program  office  to  have  elec¬ 
tronic  links  with  their  contractors  for  selected  information. 
There  are  no  technological  barriers  to  receiving  information 
directly  from  the  contractors  system  through  networking.  For 
example,  contractors  located  around  the  country  are  using  cen¬ 
tral  computers  through  networking  to  perform  many  functions. 
However,  there  are  management  barriers  that  must  be  overcome.  A 
perception  of  micro  management  can  occur  when  the  Government  PM 
has  access  to  data  at  the  same  time  or  before  contractor  senior 
management. 


Management  problems  are  not  all  on  the  contractor  side.  The 
Government  engineer  may  be  creating  problems  through  the  re¬ 
quirement  for  written  documentation  in  a  prescribed  DID  format 
rather  than  accepting  contractor  data  directly.  An  example  of 
this  is  in  the  area  of  ECP  tracking.  Contractors  have  an  auto¬ 
mated  ECP  system  which  contains  cost,  schedule  and  technical 
information.  The  PMO  requires  conversion  off  this  data  into  a 
typed  format  which  is  mailed  to  the  PMO.  The  PMO  then  converts 
the  typed  data  into  their  automated  system. 

For  cost  effective  program  management  in  an  environment  of 
expanding  technological  opportunity,  the  government  PMO  must 
tailor  a  program  with  the  contractor  that  is  open  and  free  of 
management  inhibitors,  such  as  required  formats  or  over  involve¬ 
ment.  Networking  should  allow  the  PMO  to  receive  essential  data 
from  the  contractor  at  nearly  the  same  time  as  it  is  being 
reviewed  within  their  management  structure.  This  is  especially 
true  for  the  CPR  data.  At  least  for  the  near  future  the  PM  will 
be  the  primary  link  between  the  CPR  and  TPM. 

5.5  PROPOSED  STEPS  TO  STRUCTURE  A  TPM  PROGRAM 


The  following  steps  in  development  of  a  TPM  program  are 
designed  in  the  form  of  a  checklist  to  encourage  distribution 
and  regular  review  by  government  and  contractor  PM's.  Space  has 
been  provided  for  additions,  and  as  you  have  the  opportunity  to 
utilize  and  evaluate  their  effectiveness  please  take  the  time  to 
send  your  tailored  and  improved  revisions  to:  Defense  Systems 
College,  Dept  SE-T,  TPM  Handbook  Project  Monitor,  Ft.  Belvoir 
VA  22060. 


TPM  PROGRAM 


FSD/RFP  Development 

o  System  Engineering  Management  Plan  required 
o  Formal  TPM  process  required 
o  Key  parameters  selected 
-  mission  critical 

state-of-the-art  critical 
o  TPM  reporting  requirements  identified  by  DRRB 
format 
frequency 

o  Schedule,  depth  and  documentation  of  program  status 
reviews  defined  -  MIL-STD  1521A  tailored  as  a  minimum 

Source  Selection  Plan/Conduct 

o  Evaluation  of  SEMP 

TPM  planning  addressed 
TPM  parameters  identified 

Analysis  and  forecasting  techniques  identified 
Reporting  of  TPM 

Key  events  and  TPM  milestones  identified 
TPM  implementation  procedures/schedules 
Information  flow/release  authority 
Integration  of  functional  areas 
o  System  engineering  control  of  trade-off  studies 
o  SOW  tasks  allocated  and  traceable  to  specifications 
o  Extent  of  automation  compatibility 

Negotiation 

o  Proposed  changes  (data  reduction,  schedule  changes, 
specification  changes)  reviewed  and  the  impacts  deter¬ 
mined  by  the  appropriate  functional  area  managers  before 
a  decision  is  made. 


mom 


o  SEMP  required  to  be  updated  60  days  after  contract  award 
o  Incentives  for  improved  technical  performance  evaluated 
for  system  level  effects 

After  Contract  Award 

o  Updated  SEMP  reviewed  by  PMO 

o  PM  and  key  staff  visit  contractors  facility  to  review 
contractors  engineering  system  and  management 
o  PMO  TPM  monitoring  procedures  established 

Tracking  of  delivery  dates/response  times  establish 
o  CPR  provided  to  engineering  for  analysis  as  well  a.£  ne 
business  office 

Timely  response  required 

o  Program  status  review  agendas  structured  with  contractor 
concurrence 


97 


ACRONYMS  AND  ABBREVIATIONS 


A/B 

ACWP 

AFPRO 

AFSCP 

AGB 

BCWP 

BCWS 

CAD 

CAE 

CAM 

CDR 

CDRL 

CEO 

Cl 

CPR 

C/SCSC 

C/SSR 

CWBS 

DARCOM 

DCAS 

DDR&E 

DID 

DOD 

DODD 

DODI 

DRRB 

DT&E 

D/V 

ECP 

FCA 

FQR 

FSD 

HP 

IOTS.E 

LCC 

LOE 

LP 

LRU 

MIL-STD 

MIS 

MOA 

MTBF 

NAVMAT 

NAVPRO 

OT&E 

PCA 

PDR 

PERT 

PM 

PMO 

PRO 

PRR 

PTO 


After  Burner 

Actual  Cost  of  Work  Performed 

Air  Force  Plant  Representative  Office 

Air  Force  Systems  Command  Pamphlet 

Accessory  Gear  Box 

Budgeted  Cost  for  Work  Performed 

Budgeted  Cost  for  Work  Scheduled 

Computer  Aided  Design 

Computer  Aided  Engineering 

Computer  Aided  Manufacturing 

Critical  Design  Review 

Contract  Data  Requirements  List 

Chief  Executive  Office 

Configuration  Item 

Cost  Performance  Report 

Cost/Schedule  Control  System  Criteria 

Cost/Schedule  Status  Report 

Contractor  Work  Breakdown  Structure 

Army  Material  Development  and  Readiness  Command 

Defense  Contract  Administrative  Service 

Director  of  Defense  Research  and  Engineering 

Data  Item  Description 

Department  of  Defense 

Department  of  Defense  Directive 

Department  of  Defense  Instruction 

Data  Requirements  Review  Board 

Development  Test  and  Evaluation 

Demonstration  and  Validation 

Engineering  Change  Proposal 

Functional  Configuration  Audit 

Formal  Qualification  Review 

Full  Scale  Development 

High  Pressure 

Initial  Operational  Test  and  Evaluation 

Life  Cycle  Cost 

Level  of  Effort 

Low  Pressure 

Line  Replaceable  Unit 

Military  Standard 

Management  Information  System 

Memorandum  of  Agreement 

Mean  Time  Between  Failure 

Naval  Material  Command 

Naval  Plant  Representative  Office 

Operational  Test  and  Evaluation 

Physical  Configuration  Audit 

Preliminary  Design  Review 

Program  Evaluation  and  Review  Technique 

Program  Manager 

Program  Management  Office 

Plant  Representative  Office 

Production  Readiness  Review 

Powered  Take  Off 
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R&D  Research  and  Development 

RFP  Request  for  Proposal 

SDDM  Secretary  of  Defense  Decision  Memorandum 

SDR  System  Design  Review 

SEMP  System  Engineering  Management  Plan 

SOW  Statement  of  Work 

SRR  System  Requirements  Review 

T&E  Test  and  Evaluation 

TEMP  Test  and  Evaluation  Master  Plan 

TPM  Technical  Performance  Measurement 

VEN  Variable  Exit  Nozzle 

WBS  Work  Breakdown  Structure 
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